Posted: September 16th, 2009 | Author: Jon | Filed under: Software | Tags: barry, blackberry, complaint, crash, desktop manager, evolution mail client, exchange, fix, function openfolder failed, gmail, google, imap, intellisync, internal error #4238, microsoft, mozilla thunderbird, outlook, pop3, pst file, rant, rim, settings files, sync, sync failed, the linux experiment | 5 Comments »
I fucking hate Microsoft Outlook. I cannot think up another English language sentiment that more accurately sums up my feelings regarding Microsoft’s Outlook application. As much as I hate on the Almighty Goog, I long for the days when I could use the simple, clean interface of the superbly well-designed Gmail web application. Then I went and got myself a Blackberry. All hate aside, I love my phone – it is the best phone that I’ve ever carried, and I wouldn’t consider downgrading if you paid me to do so. However, without an Exchange server, the Blackberry is inexplicably linked to Microsoft Outlook. It is the only well-supported application that the device can sync calendars and contacts with. This, in turn, forces me to use the bloated, cluttered, buggy, and altogether frustrating behemoth that is Outlook.
My current problems began with The Linux Experiment, a blog that I helped start whose purpose is to record the experiences of seven computer users with varying amounts of Linux experience who have committed to running various distributions of Linux on their primary platforms throughout the next four months. Previously, I had maintained two devices that checked my google mail account – my Blackberry, which pulled new email down from the server via the IMAP protocol, and Outlook on my Vista PC, which did the same via the POP3 protocol, and immediately deleted the messages once they came down. It was a fine balance that owed its existence to more than a few quirks in the Gmail, Blackberry, and Outlook systems, but in the end ensured that I got my email on both devices, but that it wasn’t stored on the Gmail servers, which the tinfoil-hat wearing paranoid inside of me greatly appreciated. Unfortunately, I then decided to add a third client to the mix, the Evolution client for Debian Linux, which frankly, is an extremely impressive Outlook clone that seems (initially anyway) to do some things better than Outlook itself.
In order to add a third client to the email mix, I had to remove the fine balance between IMAP and POP3 that had originally existed, and set all three devices up as IMAP clients. Further, Outlook was set to delete all messages on the server that were over 30 days old. This provided some modicum of security, while allowing all three devices to share my email. Along the way, I found out that Evolution actually has the best IMAP support of the bunch, and (unsurprisingly. If there’s on thing I’ve learned recently is that Linux does everything, and usually does it right the first time), Outlook the absolute worst that I have ever seen. For easy reference, my various complaints have been summarized into the ordered list presented below:
- IMAP folders appear outside of the “Personal Folders” area, forcing me to maintain multiple email inboxes, instead of allowing me to funnel all of my email into a single inbox. (This may be an issue common to other clients as well – I honestly don’t know).
- Outlook tends to keep IMAP connections open for too long, resulting in Gmail forcibly closing the connection, and Outlook bitching that said connection was closed by the server. There is no option (that I can find, but hey, have you looked at the option dialogs in Outlook lately?) to adjust this timeout length.
- The program does not accurately reflect message status. For example, if I receive an email on my blackberry while away from home and read it, the message status is set to read on the server, and Outlook should reflect this change. It doesn’t. Evolution does, as does the Blackberry. What the hell?
- When an email message is deleted on the Gmail server by another client, Outlook does not delete the message locally – it simply shows the message with strikethrough formatting on the subject line. In the same vein, when you delete a message in Outlook, there is no way (that I can find) to delete that message from the IMAP server so that it is reflected on other devices.
- The Linux Experiment uses a self-signed certificate to verify it’s identity to connecting mail clients. Granted, this isn’t how certificates are meant to be used, but it’s better than nothing, and we don’t have the money to pay for a CA. Outlook (as one would hope) complains that the certificate is self signed, but lacks an option to ignore this fact. In theory, this is a “feature” that notifies a user that their transaction is potentially insecure, but in practice, it’s a pain in the ass. I know that the email server has a self-signed certificate. I helped set it up. Now shut up and do your job.
Those are the big complaints about IMAP support in Outlook. I have other complaints about the application, but they’re the same as many people’s and I don’t want sore fingers, so just Bing the issue if you’re looking for a half hour rant. The point to take home is that this lackluster support is inexcusable. According to Wikipedia, the IMAP protocol has been in it’s current revision since 1996, and Gmail is hardly a fly-by-night mail server.
In any case, at the same time that I got everything set up and working between all three devices, Outlook became crash-happy, and started going down three times a day. Sometimes it would crash when I wasn’t using it at all, sometimes while I was changing account settings, occasionally when I tried to open an email, and even once while I was trying to retrieve email from the Gmail servers. The idea that Outlook (previously rock-solid stable, among it’s few good attributes) could start regularly crashing for no apparent reason whatsoever seemed far fetched. So what had changed? Well, I’d added an IMAP account and disabled a POP3 account. These changes modified the Outlook PST files (the unreadable binary blob in which the program stores everything including it’s kitchen sink), which could have potentially been corrupted in the process.
So I backed everything up, deleted my PST files, uninstalled and reinstalled Outlook. I did not realize that the program had littered my drive with settings files in both C:\Users\Username\AppData\Roaming\Microsoft\Outlook and C:\Users\Username\AppData\Local\Microsoft\Outlook, as well as (likely) numerous registry keys, and when I launched my fresh install, it attempted to read from these files, and to recreate it’s missing PST files. Balls. So I closed the application, re-deleted the newly recreated PST files, and also nuked the settings files in the two locations. Upon launching Outlook, it again somehow managed to restore all of my settings, including my RSS feeds and both of my IMAP accounts.
Fine. You restored my settings. Not the via the method that I had hoped, but the effect that I was after has been achieved. The old and possibly corrupted PST files have been recreated, my email accounts are once again being monitored by my Vista PC, and the program hasn’t crashed yet. Then I tried to sync my Blackberry with Outlook using the RIM Desktop Manager software (an application almost as poorly written as Outlook itself), and the whole house of cards came crashing down. Somehow, whatever I’d just done absolutely ruined the underlying Intellisync process, and resulted in an error that merely said “Function OpenFolder failed” with no further explanation. A quick web search resulted in nothing of value, and the sync process refused to restore my calendar and contacts from the device. The synchronization log files state only that Internal Error #4238 occurred, and that the translation of contacts failed. I Bing’d up a post on the Blackberry Forums that instructed me to delete my Intellisync folder to restore my synchronization abilities.
After following the instructions and recreating my sync profile within Desktop Manager, everything worked as expected, and my contacts and calendar were restored to Outlook. Needless to say, this entire enterprise was far more painful than I felt it should be, and only time will tell if I’ve actually fixed the crash problem, or if it will resurface in a couple of days. Regardless, I will be exploring alternatives with renewed interest. There are plenty of other email/calendar managers out there including Mozilla’s Thunderbird, which I use for my small business and absolutely love. Unfortunately, Blackberry sync is high on my list of requirements from an email client, and so far, Outlook is the only client that can do that reliably without writing a bunch of intermediary code. As a part of the Linux Experiment, I will be looking into the Barry project, which is promising, but seems to be Linux-only.
Posted: April 7th, 2009 | Author: Jon | Filed under: Education, Software | Tags: cell phones, EFS, email, Encrypting File System, enigmail, eraser, EULA, facebook, firefox, full disk encryption, gps, im, instant messaging, internet explorer, off the record, otr, outlook, pidgin, privacy, security, sms, technology, text messages, the internet, thunderbird, tor, ToS, true crypt, web surfing, windows | 3 Comments »
In short, the concept of personal privacy in communications is possibly the most important right guaranteed to those living in a free society. It is also the single most undervalued freedom in all of Western Society. The right to say what you want to whomever you want puts governing bodies to task, enables rebellion, and ensures that those wishing to sway public opinion have to work hard to demonstrate the value of the opinions that they are trying to impress upon society.
It is thus unfortunate that technology often seems to hinder our ability to ensure personal privacy – at the very least, it makes it easy to ignore the man behind the curtain. Unless one is actively aware of the risks and works to prevent them, most common methods of technological-based communication, including Facebook, cell phones, text messages, instant messaging, web surfing and email all represent massive leaks in personal privacy. These should not be taken lightly, no matter the size of your tinfoil hat. And so, in no particular order, here are some things that you should be aware of when communicating in everyday life.
- Facebook and Web 2.0 Privacy:
While this ubiquitous website has often been accused of selling your information, I find it more strange that it’s users are surprised by the idea that a website owned by a corporation would attempt to monetize the only resources immediately available to it: the information of it’s users. To me, the more scary aspect of Facebook is the slow leak of information that it inevietably causes. Like a small memory leak in an application, it isn’t a huge problem in the short term; but given time, you lose more and more control of your information as it is perused, tagged, linked to, and otherwise aggregated by the website and it’s users.
As an example, consider the following situation: You go out drinking with friends, and do something stupid. Compromising pictures are taken, uploaded, and tagged by somebody at the bar. What do you do? Well you can un-tag them, but somebody could simply replace the tag, or mention your name in the comments. Even so, anybody who recognizes you could immediately figure out who is in the photo. You can demand the image be taken down, but are reliant on the original poster complying. And even then, who is to say how many people saved local copies of the image, saw it in their news feed, viewed the gallery, or were otherwise linked before it was removed?
Simply put, Facebook is an easy way to lose control of your personal information. Consider that anybody with a developers license (which is free) has full access to this entire API from any application that they create. The incentive to create malicious trojan applications that steal and sell off information is there. The tools with which to do it are there. And the gullible users who gladly contribute thousands of dollars worth of personal information are there. So even if Facebook is ill-deserving of the allegations of selling users’ information (doubtful), any application that you add can easily present the same danger.
When using a Web 2.0 site like Facebook, Twitter, or any other site that asks the user to post personal information on a profile, it is advisable to take a quick scan through the Terms of Service (ToS), End User Licensing Agreement (EULA), and Privacy Statement (PS) of the site. Some sites, like Facebook and even Tetris Online, post outrageous claims to user data in their ToS. While I am unaware of any court case that has established a ToS, EULA, or PS as a binding legal contract between website and user, at the time of this writing, it is safe to assume that sites could attempt to act on these ‘agreements’ should user’s violate them enough times, unknowingly or otherwise.
So what can you do? Wean yourself off the koolaid. Do you really need 300 friends with whom you will never interact in real life? How many of those applications that you’ve installed and websites that you’ve joined do you really use? Is it entirely necessary to list your favourite movies, music, books, last 10 jobs, and your educational information where anybody with an internet connection could conceivably access them? I made the decision to get rid of my Facebook account a few months ago, and have never looked back. After a week, the only thing that I missed was Tetris Friends, which I later found out is available elsewhere anyway (although I haven’t registered a user account – see Jake’s comment on the Tetris Online ToS below for the reason why). If nothing else, consider taking a stroll through the myriad of options available from the settings page of your favourite social networking site and limiting the access of non-friends to your account.
- Cell Phones and GPS:
Remember the nineties movie slogan? “Shit, he’s on a cell phone. Those are untraceable!” Yeah right. By default, every cellular phone connects with two or more cell towers at any given time, and chooses the one with the strongest signal to transmit and recieve data from. This allows the phone to easily transition between towers without dropping calls while on the move. As a consequence, as long as your cellphone is on, in addition to the knowledge of what towers your phone is within range of, the phone company can locate the handset to within roughly 1-kilometer of it’s actual location by triangulation. Further, new phones often include a GPS chip that allows you to use mapping applications and geo-tag your photos. If active, the GPS chip can also transmit the location of your phone, often without your knowledge.These tracking features have many positive uses, such as enabling authorities to immediately locate the source of 9-1-1 calls, and allowing business to track the location of their employees in an effort to optimize scheduling. Unfortunately, they also have their downsides – smart phones often come with mapping applications like Google Maps that allow the user to get directions to any destination that update with respect to their location in real time. Who is to say that the almighty Goog isn’t recording all of that data, and linking it with the web-browsing habits and any email received on the same handset? While the process would certainly be undertaken in the name of increased ad-targeting abilities, having all of that relational data lying around can have dangerous side-effects if it is misused, misplaced, or simply sold.The problem of cellphone location is one inherent to the system – the ability to locate phones on the network is a natural side-effect of the way the technology works. There is little that a user can do to prevent their phone from being triangulated. GPS however, is another matter. Many phones give users the options to turn off GPS functionality, or even to limit access to the GPS radio to certain applications. Since I am paranoid, my blackberry is set to disallow Google Maps access to the GPS radio except when I explicitly allow it. This prevents the application from unintentionally spewing my location to Google without my knowledge. As previously mentioned, cameras in newer cellphones that also have GPS can record location information into images taken by the device. Users can turn this functionality off by default, which is a good idea if you intend to upload the photos to social-networking sites or other easily-accessible locations.
- SMS Text Messages and Instant Messaging:
At the end of 2007, an astounding 74% of cell phone subscribers used the SMS text messaging features of their phones. In the book ‘How to be Invisible’ by J.J. Luna, the author reveals that federal law in the USA requires that ‘all billable information [regarding a text message] be maintained for ten to fifteen years,’ including the message contents, date and time of sending and receipt, and the phone numbers of both sender and receiver. Remember the warnings about putting revealing information on the back of a postcard? The same applies to text messages, except that post cards aren’t kept on file by the postal service.Unfortunately, there is no easy solution to the SMS problem. Like post cards, users are best to simply limit what they say via these channels, as they are unencrypted, heavily logged, and contain plenty of identifying information.
The privacy situation surrounding Instant Messaging programs like MSN Messenger, Yahoo Messenger, and even IRC is in a similar state of disrepair. In order to ensure that MSN Messenger simply works on any machine, regardless of your network situation, all messages are sent from your computer, through a single connection to Microsoft servers, and then forwarded to the intended recipient(s). Messages can be logged on your machine, that of the recipient, or even at the Microsoft servers. Further, all messages are sent in unencrypted plain text that any server along the path from your computer to the recipient can log and store. By the distributed nature of the internet, the very thing that makes it so powerful, the path between any two machines generally consists of 10-15 intermediate hops. (You can check the virtual ‘distance’ between yourself and various institutions in many countries at this website.) That means that between you and your friend, there are 20-30 computers and the Microsoft servers, all of which are capable of logging anything that you say in your conversation.
Luckily, this problem is far more easily solved than that of SMS messaging. Third-party messenging clients like Pidgin allow you to connect to multiple networks (like MSN, Yahoo, Gmail, and even Facebook chat) at once, and offer an optional plugin called Off the Record (OTR) that can automatically encrypt any messages sent between you and a client that also has OTR running. It is easy to install and mindless to use, and should be a standard feature in every commercial instant messaging application. The only downside to Pidgin is that it looks ugly on Windows machines, but this is offset by it’s plugin abilities, and the fact that it can replace multiple IM clients.
- Web Surfing and Internet Connectivity:
Many people don’t realize what the act of viewing a web page actually is. When you load up this page, your computer contacts my web server, requests the page, and begins to download it to your machine, and then processes and displays the page in your web browser. That means that when you look at this page, all of it’s text, images, and other content are stored in a folder on your computer called the browser cache.
Along with the history of visited pages that many browsers keep, this information can be used by any person with access to your machine to figure out what web pages you have recently viewed. Further, many web pages leave a file behind on your computer called a ‘cookie’ that contains information that allows web sites to ‘remember’ who you are, which lets them store things like your user name and password, your preferences, or the things in your shopping cart. Again, these files can show people with access to your machine not only what sites you have recently visited, but with what account you logged into them, and potentially, what you did while logged on to the site. You can easily clear the cache, cookies and history from most browsers, or choose not to save them at all.
Additionally, because the internet is just a massive network of computers, any time you request a page, that request and all of the content that you download from the server hosting the page can travel through multiple servers, and can potentially be logged at any one of them. Further, many internet service providers keep detailed logs of your web browsing activity that authorities or unscrupulous employees can gain access to and misuse. Lastly, in the age of widespread digital piracy, many providers employ a technology called deep packet inspection to determine what your computer is uploading and downloading while connected to the internet. This technology looks inside the messages that your machine sends, determines their contents, and whether or not they should be blocked or limited. By it’s very nature, it also has the ability to snoop on any unencrypted data that you are sending, including your web requests and instant messaging conversations.
Protecting your information online is a tough thing to do. Of primary concern is the browser program that you use to view web pages. Older browsers like Microsoft’s Internet Explorer 6 have major security holes that can be used by nasty websites to steal your personal information or to install annoying programs on your computer without you doing anything out of the ordinary. Make sure that you have the latest version of your browser of choice installed. Secondly, be careful about what kind of information you give to websites. Do you really need accounts on websites that you use once a month? Putting your name or email address up on these sites can increase spam email, and lead to identity or data theft issues – all of the issues raised during the discussion about Facebook and Instant Messaging apply doubly here. For example, if searching for a job, are resume sites like Monster.ca really necessary? Putting your resume (which contains a bunch of personal information and all of your contact information) up online can lead to some devastating consequences. Finally, make sure that you have updated virus protection and firewall software installed and running on your computer at all times, and turn the machine off when you aren’t using it. If you’re really concerned about your online privacy, look into Tor, a program that encrypts your web traffic and forwards it through a bunch of random servers all over the world so that intermediate servers have no idea where the request is coming from or what data was transferred.
All email communications should be considered in the same category as Post Cards and SMS Text Messages. They are unencrypted, used worldwide, and sent through hundreds of servers that all have the ability to snoop or store copies along their journey from machine to machine. Further, webmail addresses like those available from Gmail or Windows Live store your email on their servers (sometimes indefinetly), where the messages and the information that they contain are out of your control and suceptible to snooping by authorities or unscrupulous employees.
To protect yourself while using email, you should limit the amount of sensitive business or personal information that is sent via unencrypted channels. Further, look into PGP, a (usually) free protocol that can encrypt or digitally sign all of your communications so that others cannot tamper with them. Plugins are available for most commercial email programs, although the best one that I’ve seen is the enigmail plugin for Mozilla’s Thunderbird application. Microsoft Outlook does not ship with default PGP functionality, and most of the third-party plugins that I’ve used are a pain at best and non-functional at worst. Lastly, try to limit your use of webmail, or at the very least, the amount of information that you leave on the remote servers. I use Gmail, but have Microsoft Outlook set up on my desktop which downloads all of my email, saves it locally, and deletes the copies from the server after 30 days.
- Bonus Section: Safely Deleting and Protecting your Files:
While not strictly a communication issue, many computer users don’t understand how the process of deleting a file on their computer actually works. When you delete a file on Windows, it is removed from it’s original location and sent to a folder called the Recycle Bin so that you can restore it in case you deleted it accidentally. However, even when you empty the Recycle Bin, the file is not physically removed from your machine. In order to save time, Windows simply marks the file as deleted, but never actually removes the data from your hard drive. If, at a later time, the system needs that space, it will over-write the file. But if your computer has a large hard drive that you never fill, chances are that the file can live on in the ‘empty’ space of your hard drive for years to come. Once marked deleted, many freely and commercially available programs can restore most or all of the file’s contents so long as they haven’t been overwritten by new files.
Because of this functionality, you should always assume that when you delete a file on your computer, it is for all intents and purposes, still available to anybody who cares to look for it. However, you can ensure that the file is safely deleted by using a program called a File Shredder that overwrites the file with random data, making it nearly impossible to ever recover. I would reccommend a free application called Eraser that allows you to shred any file directly from the right-click menu in windows, and can be scheduled to shred the contents of any folder or all of the free space on your hard drive at regular intervals.
A file shredder can be used to improve your security by securely deleting the contents of your recycle bin, free space on your hard drive, internet browser cache and cookie files, old email, and the porn that you downloaded that you don’t want your wife or boss to find on your machine. It’s easy to set up, integrates directly into Windows, and works without a second thought. Just beware – once a file has been shredded, it’s gone for good. Make sure that you aren’t going to need it before shredding it.
Finally, to prevent people unwarranted access to your sensitive data, look into a full-disk encryption application like TrueCrypt. It encrypts your entire hard drive and refuses anybody access until they enter a secret password that you set. Windows Passwords are good for preventing access to your machine, but if an attacker removes your hard drive and pops it into another computer, the Windows password doesn’t help you in the slightest. Full-disk encryption however, makes it extremely hard, if not impossible, to get at your files unless you personally unlock the machine.
The more expensive versions of Windows do offer a file encryption system called Encrypting File System (EFS) that can optionally encrypt your files and folders with a combination of symmetric- and public-key cryptography, similar to the system used by PGP. One potential problem with the scheme lies in the fact your Windows password and decryption password are one and the same. When you log in to Windows, the operating system transparently decrypts any files that are requested by applications. As long as you have a strong Windows password that you change every so often, this can be a good solution; however, by default, TrueCrypt encourages the use of two separate passwords and demands the first before Windows even boots, which can be far more secure (as long as the attacker chooses to boot Windows and not a separate OS from a CD or DVD drive if they get past the TrueCrypt password). Secondly, EFS does not provide full-disk encryption. Instead, it allows the user to choose which files and folders they would like to encrypt. This is generally alright, except that many programs leave digital litter around your hard drive that may not be encrypted under this scheme. For example, if you encrypt a Word document and then open it, Word can create a series of unencrypted temp files on your drive while you work on the file. Unless you wipe the free space on your drive on a regular basis, this may not be desireable when working on sensitive. If an attacker were to pull the hard drive from your machine, they could gain access to any files that you had not expressly set as encrypted by EFS. For this reason, full-disk encryption provides a better out-of-sight, out-of-mind solution that is guaranteed to protect all of your sensitive data.
Alright, if you’re still here after that massive article, I hope that you found it informative, enlightening, and easy to understand. Technology can seem tough and scary, but it doesn’t have to be that way. With a little bit of well-placed education, anybody can understand and improve the security of their communications in an effort to protect themselves from identity theft, unwanted intrusions, and overzealous authorities.
As this kind of stuff is a hobby of mine, I will be happy to answer any questions raised by or not covered by the post – leave me a comment!
Edit: Thanks to Tyler for pointing out that Enigmail for Thunderbird is an optional plugin, and is not included by default, as well as the information about Window’s EFS technology. Also thanks to Jake for pointing out the importance of reading the ToS of your favourite websites.