Ah, the Cloud. A wonderful place in the electronic ether where you can put all of your data and software so that you no longer have to manage it yourself; never mind dealing with hardware or software purchases, tech support, or IT professionals. Never mind dealing with privacy and security, avoiding vendor lock in, or being free to do what you like with your data – the cloud will take care of it all. For once, I actually agree with the viewpoint of Richard Stallman:
One reason you should not use web applications to do your computing is that you lose control… If you use a proprietary program or somebody else’s web server, you’re defenceless. You’re putty in the hands of whoever developed that software.
Stallman may be a crazy hippie, but unfortunately, he’s right. In our mad rush to create software as a service, we’ve repeatedly reinvented the wheel in an effort to coerce web browsers into doing things that desktops do with ease – and we’ve lost control over our personal data along the way. In the words of Schneier:
When a computer is within your network, you can protect it with other security systems such as firewalls and IDSs. You can build a resilient system that works even if those vendors you have to trust may not be as trustworthy as you like. With any outsourcing model, whether it be cloud computing or something else, you can’t. You have to trust your outsourcer completely. You not only have to trust the outsourcer’s security, but its reliability, its availability, and its business continuity.
Even though living in the cloud may look great on paper – “All of my services are served by Google, and available via a single user account!” – what happens if the Almighty Goog goes out of business tomorrow? Or just shuts down Google Docs? God knows, it isn’t making any money off of the service. Amazon’s S3 and EC2 services are no better, with rare, but sometimes lengthy outages that can negatively effect many online businesses that rely on the services being running.
The point that I’m trying to get at with all of this ranting and raving is that nobody owns your data but you. How many times have you been told to back up your hard drive? The same rules apply (if not doubly so) when talking about data stored ‘in the cloud’. There is little incentive for the vendor to care about what it does with the data of users who get its service for free. Remember Schofield’s Second Law of Computing:
Data doesn’t really exist unless you have two copies of it. Preferably more. And the only person who can be held responsible for that is you.
The internet is a magical place, and has changed our world in inumerable ways. In this video, Kevin Kelly dissects the accomplishments of the first ’5000 days’ of the World Wide Web, and makes some startling predictions for the next 5000. Ultimately, for any of his ideas to come to fruition, we users will need to surrender much of the control over our data to faceless companies motivated solely by profit. I’m for crafty capitalism as much as the next guy – hell, I want to make my living in this industry – but is this really how we want it to go down?
A lot of the programming that I do for myself involves the creation of little tools that patch two existing parts of my workspace together so that they work better. For example, I recently had need of an SFTP client that links into a database front end to retrieve a list of files to download. Since the SFTP specification is ridiculously long and convoluted, I decided to try and use WinSCP to take care of the FTP portion of the program instead of re-inventing the wheel. Because I was trying to patch the database layer and FTP client together into a single application, I decided to take a shot at controlling WinSCP directly from the command line so that the two portions of the project were contained in a single application.
Readers who have spent time fooling around in Visual Basic 6 might remember the Shell function that allows a program to launch another process. Generally speaking, if we don’t care about the output of the program, or are just launching something like a browser window, this command is more than sufficient. However in this case, I wanted to capture the output of WinSCP and its exit code so that I could tell if errors had occurred and take appropriate action on them, so I needed something with a little bit more kick.
Since I wrote my code with WinSCP in mind, the rest of the article is going to focus on creating an application that will serve as an FTP front-end that allows the user to upload or download a file to or from a remote SFTP server. That said, the ideas are general and easy to adapt to any other application that supports command-line arguments.
You’ll want to begin by dragging and dropping the executable that you wish to automate directly into your project, and setting its “Copy to Output” property to “Copy to Newer” to ensure that your application can always find it. For WinSCP, that included both the winscp.com and winscp.exe files from the install directory. That done, let’s jump in.
Using System.Diagnostics.Process:
This handy System class provides a beefier version of the Shell functionality, and is really very easy to use, once you get the hang of it. At the top of your class, add the line imports System.Diagnostics so that we can find the pieces that we’ll need with ease.
When automating a command-line application, we first create a ProcessStartInfo object that contains all of the information about the process that we would like to create. Start by declaring one of these:
Dim startInfo As New ProcessStartInfo("winscp.com")
This line creates a ProcessStartInfo object with the path to the application that we’d like to automate as it’s only argument. Next, we set a few properties of the object:
The first line passes the string commandLineArguments to startInfo, telling it to give them to winscp.com as command line arguments. Command line argument options for most programs can be found on their websites. If you’re interested in WinSCP in particular, it has some great online documentation that you can take a look at. The next three lines tell the startInfo object that it should direct the standard output from the process to a System.IO.StreamReader object where we can intercept it, and that it should run that process in the background, without showing the window to the user.
Next, we create a new Process object, and pass it the information that we just stored in startInfo:
Dim p as New Process p = Process.Start(startInfo)
This creates a new process for winscp.com, and passes it the command line arguments that we stored earlier in startInfo. Now, we need to catch the standard output from our newly created process and store it somewhere so that our users can see what the application did.
Dim s as IO.StreamReader = p.StandardOutput While p.Responding Try lstOutput.Items.Add(s.ReadLine) Catch Exit While End Try End While
On the first line, we create a new IO.StreamReader object that will catch the standard output from our hosted process. The while loop tries repeatedly to read a line of output from the application and dump it into a list box while the process is still responding to the operating system. When the application finishes its work, s.ReadLine will return null, killing the loop and allowing the program to continue.
Finally, when a program exits, it returns an exit code that lets the operating system know whether or not it completed its work successfully. We can access this code with the line
p.ExitCode
In most programs, if this returns zero, the program finished successfully. If an error was encountered, this function should return something other than zero, and depending on the program, the other value may have some meaning that you can use as a status check.
Next Steps:
Because this code is purposely general, it should be easy to adapt it to control most any application that provides some kind of command line interface. This method can make for a nice alternative to batch scripts if you need to write batch scripts with some kind of logic in them and don’t feel like installing a python interpreter to get the job done. At the cost of making the code application-specific, you could easily change it to analyze the output from the program and take some action based on those return values.
Downloads:
As always, the source is available for download. Check out Tyler Burton’s Hash Verifier application if you want to ensure that you’re getting the same copy of the code that I claim you are. Source Code: Click Here MD5 Hash: 4B5D543E193CAC2B4B9477727E17EFB2 SHA1 Hash: ABD8692CBB20D316DBF35FE650399D034DA57697
With nothing important to do this morning (except for all that studying that I should be tackling), I decided to play a favourite game of mine – 1701 A.D. It’s an excellent RTS-style game from Germany in which you play the role of an explorer in the new world. It is your task to build a successful settlement that supports itself and glorifies the Queen. Think SimCity meets Age of Empires, all rendered in pretty graphics and with engrossing game play that will easily make your entire day fly right by. Unless, of course, you try to install and run the game on Windows Vista:
Alright, so, the installer must have failed. I went and ran DrvSetup_x64.exe from the game disc, and got another fantastic error message:
Well, that’s handy. A Google and a half later, I found a slew of angryposts about Tages copy protection on various message boards. I gather that Tages essentially installs a copy-protection driver to your system that isn’t signed by Microsoft. This is an issue under Vista, and the system refuses to install the driver for security reasons. You can reboot and tell Vista to not require that drivers be signed by tapping F8 at boot and selecting “Disable Driver Signature Enforcement” from the boot menu, but according to these posts, you have to perform this action every time you wish to play the game, which is a big security hole as well as a big pain in the ass.
I found a link to an updated version of the drivers at the Tages website that were signed by Microsoft, and installed with no issues, finally allowing me to run a game that I legitimately paid for instead of simply pirating. While I appreciate the situation that studios and distributors find themselves in regarding piracy, I can’t help but be indignant when confronted with copy protection schemes that punish the honest customer. Between my mother and I, our family has purchased three copies of this game, and every previous installment of the franchise. She wouldn’t have been able to figure out the issue with the drivers. Why weren’t the drivers distributed on the game disc signed by Microsoft? Why did the game not automatically update the drivers after validating a legitimate install? Why was there no post about this issue on the game website?
I have to admit that while looking for a solution, I did consider torrenting a cracked version of the game instead of bothering to fix my legitimate copy, which is funny, because the Tages website states that:
The proof of TAGES™’s effectiveness is undeniable as illustrated on various web sites. All major competitors have been hacked and the hackers have made generic cracks available for free. Anyone can break into these systems and produce illegal copies. With TAGES™ there will never be a generic crack, and there will never be one-to-one copies. It is physically impossible.
An interesting claim, since a quick Google search shows quite a few illegal versions of the game available for torrent. At the end of the day, while I cannot condone piracy, this kind of nonsense really gets me angry. I understand that piracy is pushing developers away from the PC platform and toward the more secure Console systems. I understand that piracy can destroy the user experience of a game; but I demand that if a company is going to take steps to prevent piracy, they do it well, and don’t inhibit the actions of legitimate users of paid copies of their product. You wouldnt knowingly push a game to market that had a bug that made it unplayable – so why do DRM schemes get a free pass?
In short, the concept of personal privacy in communications is possibly the most important right guaranteed to those living in a free society. It is also the single most undervalued freedom in all of Western Society. The right to say what you want to whomever you want puts governing bodies to task, enables rebellion, and ensures that those wishing to sway public opinion have to work hard to demonstrate the value of the opinions that they are trying to impress upon society.
It is thus unfortunate that technology often seems to hinder our ability to ensure personal privacy – at the very least, it makes it easy to ignore the man behind the curtain. Unless one is actively aware of the risks and works to prevent them, most common methods of technological-based communication, including Facebook, cell phones, text messages, instant messaging, web surfing and email all represent massive leaks in personal privacy. These should not be taken lightly, no matter the size of your tinfoil hat. And so, in no particular order, here are some things that you should be aware of when communicating in everyday life.
Facebook and Web 2.0 Privacy:
While this ubiquitous website has oftenbeenaccused of selling your information, I find it more strange that it’s users are surprised by the idea that a website owned by a corporation would attempt to monetize the only resources immediately available to it: the information of it’s users. To me, the more scary aspect of Facebook is the slow leak of information that it inevietably causes. Like a small memory leak in an application, it isn’t a huge problem in the short term; but given time, you lose more and more control of your information as it is perused, tagged, linked to, and otherwise aggregated by the website and it’s users.
As an example, consider the following situation: You go out drinking with friends, and do something stupid. Compromising pictures are taken, uploaded, and tagged by somebody at the bar. What do you do? Well you can un-tag them, but somebody could simply replace the tag, or mention your name in the comments. Even so, anybody who recognizes you could immediately figure out who is in the photo. You can demand the image be taken down, but are reliant on the original poster complying. And even then, who is to say how many people saved local copies of the image, saw it in their news feed, viewed the gallery, or were otherwise linked before it was removed?
Simply put, Facebook is an easy way to lose control of your personal information. Consider that anybody with a developers license (which is free) has full access to this entire API from any application that they create. The incentive to create malicious trojan applications that steal and sell off information is there. The tools with which to do it are there. And the gullible users who gladly contribute thousands of dollars worth of personal information are there. So even if Facebook is ill-deserving of the allegations of selling users’ information (doubtful), any application that you add can easily present the same danger.
When using a Web 2.0 site like Facebook, Twitter, or any other site that asks the user to post personal information on a profile, it is advisable to take a quick scan through the Terms of Service (ToS), End User Licensing Agreement (EULA), and Privacy Statement (PS) of the site. Some sites, like Facebook and even Tetris Online, post outrageous claims to user data in their ToS. While I am unaware of any court case that has established a ToS, EULA, or PS as a binding legal contract between website and user, at the time of this writing, it is safe to assume that sites could attempt to act on these ‘agreements’ should user’s violate them enough times, unknowingly or otherwise.
So what can you do? Wean yourself off the koolaid. Do you really need 300 friends with whom you will never interact in real life? How many of those applications that you’ve installed and websites that you’ve joined do you really use? Is it entirely necessary to list your favourite movies, music, books, last 10 jobs, and your educational information where anybody with an internet connection could conceivably access them? I made the decision to get rid of my Facebook account a few months ago, and have never looked back. After a week, the only thing that I missed was Tetris Friends, which I later found out is available elsewhere anyway (although I haven’t registered a user account – see Jake’s comment on the Tetris Online ToS below for the reason why). If nothing else, consider taking a stroll through the myriad of options available from the settings page of your favourite social networking site and limiting the access of non-friends to your account.
Cell Phones and GPS:
Remember the nineties movie slogan? “Shit, he’s on a cell phone. Those are untraceable!” Yeah right. By default, every cellular phone connects with two or more cell towers at any given time, and chooses the one with the strongest signal to transmit and recieve data from. This allows the phone to easily transition between towers without dropping calls while on the move. As a consequence, as long as your cellphone is on, in addition to the knowledge of what towers your phone is within range of, the phone company can locate the handset to within roughly 1-kilometer of it’s actual location by triangulation. Further, new phones often include a GPS chip that allows you to use mapping applications and geo-tag your photos. If active, the GPS chip can also transmit the location of your phone, often without your knowledge.These tracking features have many positive uses, such as enabling authorities to immediately locate the source of 9-1-1 calls, and allowing business to track the location of their employees in an effort to optimize scheduling. Unfortunately, they also have their downsides – smart phones often come with mapping applications like Google Maps that allow the user to get directions to any destination that update with respect to their location in real time. Who is to say that the almighty Goog isn’t recording all of that data, and linking it with the web-browsing habits and any email received on the same handset? While the process would certainly be undertaken in the name of increased ad-targeting abilities, having all of that relational data lying around can have dangerous side-effects if it is misused, misplaced, or simply sold.The problem of cellphone location is one inherent to the system – the ability to locate phones on the network is a natural side-effect of the way the technology works. There is little that a user can do to prevent their phone from being triangulated. GPS however, is another matter. Many phones give users the options to turn off GPS functionality, or even to limit access to the GPS radio to certain applications. Since I am paranoid, my blackberry is set to disallow Google Maps access to the GPS radio except when I explicitly allow it. This prevents the application from unintentionally spewing my location to Google without my knowledge. As previously mentioned, cameras in newer cellphones that also have GPS can record location information into images taken by the device. Users can turn this functionality off by default, which is a good idea if you intend to upload the photos to social-networking sites or other easily-accessible locations.
SMS Text Messages and Instant Messaging: At the end of 2007, an astounding 74% of cell phone subscribers used the SMS text messaging features of their phones. In the book ‘How to be Invisible’ by J.J. Luna, the author reveals that federal law in the USA requires that ‘all billable information [regarding a text message] be maintained for ten to fifteen years,’ including the message contents, date and time of sending and receipt, and the phone numbers of both sender and receiver. Remember the warnings about putting revealing information on the back of a postcard? The same applies to text messages, except that post cards aren’t kept on file by the postal service.Unfortunately, there is no easy solution to the SMS problem. Like post cards, users are best to simply limit what they say via these channels, as they are unencrypted, heavily logged, and contain plenty of identifying information.
The privacy situation surrounding Instant Messaging programs like MSN Messenger, Yahoo Messenger, and even IRC is in a similar state of disrepair. In order to ensure that MSN Messenger simply works on any machine, regardless of your network situation, all messages are sent from your computer, through a single connection to Microsoft servers, and then forwarded to the intended recipient(s). Messages can be logged on your machine, that of the recipient, or even at the Microsoft servers. Further, all messages are sent in unencrypted plain text that any server along the path from your computer to the recipient can log and store. By the distributed nature of the internet, the very thing that makes it so powerful, the path between any two machines generally consists of 10-15 intermediate hops. (You can check the virtual ‘distance’ between yourself and various institutions in many countries at this website.) That means that between you and your friend, there are 20-30 computers and the Microsoft servers, all of which are capable of logging anything that you say in your conversation.
Luckily, this problem is far more easily solved than that of SMS messaging. Third-party messenging clients like Pidgin allow you to connect to multiple networks (like MSN, Yahoo, Gmail, and even Facebook chat) at once, and offer an optional plugin called Off the Record (OTR) that can automatically encrypt any messages sent between you and a client that also has OTR running. It is easy to install and mindless to use, and should be a standard feature in every commercial instant messaging application. The only downside to Pidgin is that it looks ugly on Windows machines, but this is offset by it’s plugin abilities, and the fact that it can replace multiple IM clients.
Web Surfing and Internet Connectivity:
Many people don’t realize what the act of viewing a web page actually is. When you load up this page, your computer contacts my web server, requests the page, and begins to download it to your machine, and then processes and displays the page in your web browser. That means that when you look at this page, all of it’s text, images, and other content are stored in a folder on your computer called the browser cache.
Along with the history of visited pages that many browsers keep, this information can be used by any person with access to your machine to figure out what web pages you have recently viewed. Further, many web pages leave a file behind on your computer called a ‘cookie’ that contains information that allows web sites to ‘remember’ who you are, which lets them store things like your user name and password, your preferences, or the things in your shopping cart. Again, these files can show people with access to your machine not only what sites you have recently visited, but with what account you logged into them, and potentially, what you did while logged on to the site. You can easily clear the cache, cookies and history from most browsers, or choose not to save them at all.
Additionally, because the internet is just a massive network of computers, any time you request a page, that request and all of the content that you download from the server hosting the page can travel through multiple servers, and can potentially be logged at any one of them. Further, many internet service providers keep detailed logs of your web browsing activity that authorities or unscrupulous employees can gain access to and misuse. Lastly, in the age of widespread digital piracy, manyproviders employ a technology called deep packet inspection to determine what your computer is uploading and downloading while connected to the internet. This technology looks inside the messages that your machine sends, determines their contents, and whether or not they should be blocked or limited. By it’s very nature, it also has the ability to snoop on any unencrypted data that you are sending, including your web requests and instant messaging conversations.
Protecting your information online is a tough thing to do. Of primary concern is the browser program that you use to view web pages. Older browsers like Microsoft’s Internet Explorer 6 have major security holes that can be used by nasty websites to steal your personal information or to install annoying programs on your computer without you doing anything out of the ordinary. Make sure that you have the latestversion of your browser of choice installed. Secondly, be careful about what kind of information you give to websites. Do you really need accounts on websites that you use once a month? Putting your name or email address up on these sites can increase spam email, and lead to identity or data theft issues – all of the issues raised during the discussion about Facebook and Instant Messaging apply doubly here. For example, if searching for a job, are resume sites like Monster.ca really necessary? Putting your resume (which contains a bunch of personal information and all of your contact information) up online can lead to some devastating consequences. Finally, make sure that you have updated virus protection and firewall software installed and running on your computer at all times, and turn the machine off when you aren’t using it. If you’re really concerned about your online privacy, look into Tor, a program that encrypts your web traffic and forwards it through a bunch of random servers all over the world so that intermediate servers have no idea where the request is coming from or what data was transferred.
Email: All email communications should be considered in the same category as Post Cards and SMS Text Messages. They are unencrypted, used worldwide, and sent through hundreds of servers that all have the ability to snoop or store copies along their journey from machine to machine. Further, webmail addresses like those available from Gmail or Windows Live store your email on their servers (sometimes indefinetly), where the messages and the information that they contain are out of your control and suceptible to snooping by authorities or unscrupulous employees.
To protect yourself while using email, you should limit the amount of sensitive business or personal information that is sent via unencrypted channels. Further, look into PGP, a (usually) free protocol that can encrypt or digitally sign all of your communications so that others cannot tamper with them. Plugins are available for most commercial email programs, although the best one that I’ve seen is the enigmail plugin for Mozilla’s Thunderbird application. Microsoft Outlook does not ship with default PGP functionality, and most of the third-party plugins that I’ve used are a pain at best and non-functional at worst. Lastly, try to limit your use of webmail, or at the very least, the amount of information that you leave on the remote servers. I use Gmail, but have Microsoft Outlook set up on my desktop which downloads all of my email, saves it locally, and deletes the copies from the server after 30 days.
Bonus Section: Safely Deleting and Protecting your Files: While not strictly a communication issue, many computer users don’t understand how the process of deleting a file on their computer actually works. When you delete a file on Windows, it is removed from it’s original location and sent to a folder called the Recycle Bin so that you can restore it in case you deleted it accidentally. However, even when you empty the Recycle Bin, the file is not physically removed from your machine. In order to save time, Windows simply marks the file as deleted, but never actually removes the data from your hard drive. If, at a later time, the system needs that space, it will over-write the file. But if your computer has a large hard drive that you never fill, chances are that the file can live on in the ‘empty’ space of your hard drive for years to come. Once marked deleted, many freely and commercially available programs can restore most or all of the file’s contents so long as they haven’t been overwritten by new files.
Because of this functionality, you should always assume that when you delete a file on your computer, it is for all intents and purposes, still available to anybody who cares to look for it. However, you can ensure that the file is safely deleted by using a program called a File Shredder that overwrites the file with random data, making it nearly impossible to ever recover. I would reccommend a free application called Eraser that allows you to shred any file directly from the right-click menu in windows, and can be scheduled to shred the contents of any folder or all of the free space on your hard drive at regular intervals.
A file shredder can be used to improve your security by securely deleting the contents of your recycle bin, free space on your hard drive, internet browser cache and cookie files, old email, and the porn that you downloaded that you don’t want your wife or boss to find on your machine. It’s easy to set up, integrates directly into Windows, and works without a second thought. Just beware – once a file has been shredded, it’s gone for good. Make sure that you aren’t going to need it before shredding it.
Finally, to prevent people unwarranted access to your sensitive data, look into a full-disk encryption application like TrueCrypt. It encrypts your entire hard drive and refuses anybody access until they enter a secret password that you set. Windows Passwords are good for preventing access to your machine, but if an attacker removes your hard drive and pops it into another computer, the Windows password doesn’t help you in the slightest. Full-disk encryption however, makes it extremely hard, if not impossible, to get at your files unless you personally unlock the machine.
The more expensive versions of Windows do offer a file encryption system called Encrypting File System (EFS) that can optionally encrypt your files and folders with a combination of symmetric- and public-key cryptography, similar to the system used by PGP. One potential problem with the scheme lies in the fact your Windows password and decryption password are one and the same. When you log in to Windows, the operating system transparently decrypts any files that are requested by applications. As long as you have a strong Windows password that you change every so often, this can be a good solution; however, by default, TrueCrypt encourages the use of two separate passwords and demands the first before Windows even boots, which can be far more secure (as long as the attacker chooses to boot Windows and not a separate OS from a CD or DVD drive if they get past the TrueCrypt password). Secondly, EFS does not provide full-disk encryption. Instead, it allows the user to choose which files and folders they would like to encrypt. This is generally alright, except that many programs leave digital litter around your hard drive that may not be encrypted under this scheme. For example, if you encrypt a Word document and then open it, Word can create a series of unencrypted temp files on your drive while you work on the file. Unless you wipe the free space on your drive on a regular basis, this may not be desireable when working on sensitive. If an attacker were to pull the hard drive from your machine, they could gain access to any files that you had not expressly set as encrypted by EFS. For this reason, full-disk encryption provides a better out-of-sight, out-of-mind solution that is guaranteed to protect all of your sensitive data.
Alright, if you’re still here after that massive article, I hope that you found it informative, enlightening, and easy to understand. Technology can seem tough and scary, but it doesn’t have to be that way. With a little bit of well-placed education, anybody can understand and improve the security of their communications in an effort to protect themselves from identity theft, unwanted intrusions, and overzealous authorities.
As this kind of stuff is a hobby of mine, I will be happy to answer any questions raised by or not covered by the post – leave me a comment!
Cheers,
Jonathan
Edit: Thanks to Tyler for pointing out that Enigmail for Thunderbird is an optional plugin, and is not included by default, as well as the information about Window’s EFS technology. Also thanks to Jake for pointing out the importance of reading the ToS of your favourite websites.
Some who read me on a regular basis might be wondering what ever happened to the RPG I was writing in VisualStudio using Microsoft’s XNA technology. Truth is, it never went anywhere. Like a lot of past projects, I got very psyched about it, wrote a bunch of code, and then school started and I got way too busy to finish it. I haven’t touched the code since I wrote that post, but now I’ve got some new ideas.
Last term in school, I took an excellent course on OpenGL programming with C++. The course covered how to create and render 3D graphics, with study lent to topics such as window management, points, vertices, and polygons, lighting and shading, hidden surface removal, and texturing. My earlier post about the MAX 3D Engine was a byproduct of that course.
Since then, I’ve discovered a managed .NET wrapper around the OpenGL libraries called the Tao Framework that allows you to (in theory) code any graphics application in Visual Basic or C# that you could in C++, with the added bonus of the pretty IDE, code completion, top of the line window handling procedures, and the .NET libraries. Now at this point, if you’re a graphics programmer, you’re laughing aloud at my outrageous claim – managed, run-time interpreted code could never be fast enough to run a video game! You may be correct. Frankly, I have no idea, as I haven’t yet had the time to write a full video game.
What I have come up with however, are two starter projects for anybody wishing to try their hand at OpenGL programming using Tao and Visual Basic.NET. The code in both is well documented, easy to follow (especially if you are familiar with standard OpenGL routines), and seems to run at a reasonable 60fps. Now, I can’t tell until I add a few more polygons to the scene whether this framerate is an artificial limit applied by the environment, or if interpreted code actually has no hope of ever running a game at a reasonable speed. That is an experiment for a later day. For now, I will simply share these starter projects for all to use. If you do something with them, please leave a comment and let me know how it went.
Tao2D Test Harness:
A simple application that spins a tri-coloured, smooth-shaded triangle around the y-axis.
After being linked to the website by a colleague today, I spent a few hours rediscovering my love for TED Talks. For those who have never been, the website is like youtube, but without all the garbage. It is simply a massive collection of lectures by really smart people on just about every topic under the sun.
This evening, I stumbled upon a particularly amazing talk by Ed Ulbrich, the visual effects executive producer on The Curious Case of Benjamin Button. He and His team at Digital Domain spent four years creating the digital head that plays the part of Benjamin Button for the first half of the film. Most importantly, they didn’t animate any of it.
The team developed a technology called Emotion Capture that allowed them to analyze video of Brad Pitts’ facial performance of a scene and map that directly onto their computer models of his face. For anybody interested in computer graphics, this is an incredible lecture, and a must see.
More information regarding Digital Domain and the technology that they developed for this project can be found here and here.
Readers will recall the issues that I’d had some time ago in trying to force Windows Backup to play ball with an external hard drive encrypted by TrueCrypt. For some reason or another, Windows Backup refused to recognize the mounted drive as a valid backup location. One reader recommended that I try Acronis True Image out. Seeing as I like free stuff, I’ve found a free solution that solves the problem entirely.
Now it should be said from the outset that this process is a little bit ugly and a tad long winded, but that it does a really swell job and is full of opportunities to improve upon and to customize it for your particular situation.
This morning I found an excellent article called PC Game Piracy Examined written by Koroush Ghazi over at TweakGuides.com that addresses the current state of PC video game piracy, and the so called “death of PC gaming” theory that has been circulating the internet as of late. Readers will recall my DRM essay uploaded some months ago that concluded among other things that
The basic flaws inherent in every DRM system, the rampant destruction of customers right to first sale and fair use, and the fear of device lock-in and market stagnation lead this author to believe that a better solution need be proposed to protect the intellectual properties of media producers.
Upon reading through Ghazi’s lengthy ten page article, I have to admit that I’ve been forced to take a hesitant step backward and re-evaluate some of the opinions that I’ve long held regarding DRM technologies. While I still agree and stand by many of the points made in my essay, his writing has cleverly challenged some of my conclusions.
The concept of Zero-Day Piracy is one that I did not research when writing my essay, and deals with the idea that hype for any product is at its height on launch day. Many expectant fans will rush to obtain the product by whatever means possible as soon as they can, and if a pirated version of that product is available on or before launch day, the first spike of revenue (often what ensures that the studio breaks even and continues to release subsequent products) is lost. In this regard, producers realize that their products will eventually be pirated, and use DRM to ensure that their products stand available only in legitimately purchased form for at least a week after launch. While it sounds hopeless, this week of legitimate sales is instrumental in ensuring the economic success of the product.
Another part of the article that really resonated with me was the discussion of how game developers and publishers are changing their business models to adapt to the problem of mass piracy. Some are choosing DRM, while others are moving toward MMORPG’s with a subscription model, and some are moving away from the PC entirely. As a portion of this section, the author discussed things that developers and publishers can do to try and reduce the number of pirated copies of their games. One of these suggestions called strongly for more demos. I especially appreciate this, as for some reason graphics-heavy games tend to crash my video card drivers on a regular basis (I blame Vista and/or Ultramon), and I like to try a game out to ensure that it will in fact run stable on my system before putting my money down. I had this problem after purchasing Left 4 Dead the other day ($50 later, it’s almost unplayable), and thought twice about buying Far Cry 2 this morning, realizing that it would probably push my machine too far, even though my video card is listed as supported. A demo of that game would have eased my choice along considerably.
While I still believe that DRM is bad for the system in the long run, Ghazi successfully pounds the point home that publishers turn to DRM systems in an effort to offset the damage done to their business by piracy. To excuse piracy on the basis of clever DRM systems that are hard to bypass is to turn a blind eye to the destruction reaped upon the digital marketplace by that same act of piracy and to in effect lend oneself to the continued downfall of beloved products, including music, movies, and video games.
I would recommend that any person who has in the past participated in the piracy of software, movies, or music take the time to read through this article, lengthy though it may be. Entertainment distribution is in a tough place across the board these days, but PC game development in particular is a sector that many inspired artists are dropping out of all together because they can no longer afford the risks involved. These same risks can make it nearly impossible to break into the business as an independent developer, as evidenced by the piracy rate documented by 2D Boy, developers of the incredible title World of Goo. In effect, by continuing to pirate products that we know and love, gamers are contributing to the downfall of this valued form of entertainment, and whether we agree with DRM or not, I think that it is important that we all take a moment to consider the real entertainment value of these products that we take for granted, and consider the true implications of our actions.
Last term, I took the CP411 Computer Graphics course at Wilfrid Laurier from Professor Hongbing Fan. It was an introductory course to creating computer graphics systems using standard C++ and OpenGL technologies. All in all, it was probably the best course I’ve taken yet at WLU, and I learned an awful lot. As a final project, we were asked to create some 3D application, and I chose (without really considering the ramifications of such a commitment) to attempt a 3D graphics engine.
The result was MAX; a highly extensible, working 3D graphics engine that I would say is about par with an N64 in terms of graphics capabilities, and leaves enough room for improvement that I could probably spend another full year working solely on it. However, for 2 weeks development time, I think that what I came up with represents an excellent baseline for what is necessary of a 3D engine, and that it was a worthwhile project and a great learning experience.
Following are a number of screen shots from the project:
The engine itself, while simplistic, is robust. It relies on OpenGL for its lighting and backface removal functions, supports mouse and keyboard input, texturing, models (although I haven’t written any adapters for importing complex model formats yet – all of the models are hard-coded for now), collision detection, and basic physics such as gravity.
By and large, I had problems with the amount of work. When I decided to create a 3D engine, I made the mistake of not setting out some boundaries for what I wanted to get out of it. I had 2 weeks to work on the project, and just started throwing whatever I had at the project. This of course led to feature creep, poor organization, and a continuous reduction in expectations for the final product. That said, I really am proud of the result of the experiment, although I would definetly change my work habits next time around.
Finally, here is a quick list of improvements that I feel need to be made to the engine before it is ready to host any kind of game:
Improved Collision Detection: Current algorithms utilize a simplified object-aligned bounding box algorithm that simply checks whether any two bounding boxes intersect with each other. Pros of the method are that multiple bounding boxes are supported for any model, and that moving model collisions (colliders) are abstracted from non-moving model collisions (colidees), meaning that no two inanimate objects will ever be checked for a collision with each other. A better method is to project the model onto each of the axes, and check for collisions between the resulting 2D shapes. If the 2D shapes collide on every axis, then the two objects are colliding. For an awesome tutorial (albeit in flash script and for a 2D engine, but easily extensible, check out N Tutorials)
Object Collision Events: Currently, when any two objects in the game collide, they both stop in their tracks. This works, but it would be better to assign some scriptable action to this event, and to implement semi-complex interactions like friction and force and the ability for one object to push another. The engine already supports gravity on all moveable objects, as well as collision detection with the outer walls of the world area, so a little bit more physics would be nice.
Complex Model Support: This is a seriously trying thing to do. Models of more than about 50 polygons quickly become unwieldy to handle quickly in memory and require some fast code to display at a decent frame rate. At least that’s what I’ve read. Since I needed an engine fast, and didn’t want to waste my time modeling (heck, I can’t draw anyway), I created my models as hard-coded in-engine entities comprised of simplistic geometry and this did not become a concern. However, I would like to improve my model class to support a greater number of polygons and textures, and enable reading in from common model formats.
Texture and Lighting Class Wrappers: Currently these items are exposed right in the main class of the application. It would be nice to have some sort of abstraction around them that allows for simple palette and lighting switches, as well as for improved resource management. Also, dynamic shadows are cool, and require only one additional rendering pass that should be easy enough to implement.
Particle Engine and Visual Effects: An easy way to do really cool visual effects is with a simple, sturdy particle engine that supports textures, anti-aliasing, and swarm-like movement of the particles. This can be used for bugs, birds, magic spells, weather effects, leaves in the wind, tornadoes, or really most any other simple effect you can think of. NeHe has an interesting tutorial on the matter here.
Overall, I’m proud of what I accomplished in such a short period of time. As always, the code is available for download for those who wish to play with it or just check it out.
Enjoy,
Jon
Edit: Resized the images so that they aren’t all gross and stretched out. Also, my apologies to any non-programmers attempting to download the engine – all you’ll get is Dev C++ source code. I’ll compile it and up an *.exe file tonight for those who aren’t interested in the pain and anguish involved in attempting to use Dev.
I listen to a whole lot of music. There are really no two ways about it; if I’m at home, music is playing on my computer. If I’m not, it’s playing on one of the myriad of portable devices I own. I maintain a music library with just over ten thousand songs spanning most every genre and year in the last half-century. This of course means that I need to run some mean music organization software.
Like most other iPod owners, my default player is Apple’s iTunes, but lately it just hasn’t been impressing me as much as it used to, as evidenced by this long list of complaints:
Regular memory usage is around 130MB, and that number balloons to well over 250MB if I dare open that new-fangled glossy cover flow feature.
To make matters worse, if I close the cover flow, usage drops almost immediately to around 180MB, but refuses to dip lower, making me suspect a memory leak.
I keep the application running 24 hours a day and after about a week of uptime, its footprint can often climb above 300MB (without coverflow), lending more evidence to the memory leak theory.
As I type this, iTunes is converting some WMA tracks to MP3 while playing music, and is sipping 78% of my 2GHz dual core processor. That’s inexcusable.
Smart playlists are dumb at best, allowing only a global AND or a global OR for all playlist conditions. Sometimes this just doesn’t cut it, and I find myself chaining two or more playlists or using De Morgan’s law just to figure out the boolean logic behind a desired set of conditions. Yeah, I’m that much of a nerd.
The iTunes store peddles DRM-laden garbage. Sure, they sell iTunes Plus tracks now, but those are still m4a files (while the rest of the world sells MP3), and most big-label releases are still protected by FairPlay DRM.
When importing a folder full of songs, it often creates two copies of each track in my library and on my file system. Except that sometimes I get two copies of only some of the tracks, while the others copy as normal…
If a friend who runs a Mac brings his iPod over, I can’t plug that iPod in and stream tracks off of it because it’s Mac formatted. Sure, Windows doesn’t know the format of that hard drive, but Apple does; couldn’t they write a driver layer that can read it? I’ll bet that they could.
iPod cables are expensive. What are all of those damned pins used for? My blackberry is a mobile computer and it syncs just fine with a USB cable. Every device on the market uses the USB standard, while Apple sticks to this ridiculous cable with at least 30 pins on it, forcing third party manufacturers to license the design and jacking up accessory cost as a result (ok, this is really an iPod complaint, sue me).
Lastly, when editing information for multiple tracks, like for an entire album at once, the application doesn’t save my changes to the id3 tags. Yet if i modify the info on each track separately, it does. What gives, Steve?
Overall, considering the long list of features that iTunes does provide, it’s certainly not the worst program ever written, but I can’t help but think that the Windows versions an after thought in Cupertino and that they just don’t get as much polish as the Mac versions of the program. Long story short, I recently went looking for alternatives. The shortlist of what I require from a jukebox application:
Easy to navigate interface that lets me search, organize, and find my music with ease
Track ratings. This is a must when your library has more breadth than the weekly Top 40 list.
Smart and Static playlists that let me automatically partition my music into logical subsets
Equalizers are nice, but should come with some presets that i can tweak to my setup.
Low memory footprint with no leaks – I run this app all day long, so make it efficient
Automatic file system organization so that I don’t have to worry about it.
Search by track name, artist, album, etc.
CD burning and ripping is a nice extra, but I can use a third-party app with no complaints if necessary
XML file importing so that I can migrate my giant library from my existing solution is a must. Honestly, I’ve written a number of simple apps that parse an exported iTunes library (see the sidebar); This is a dead simple feature to add.
And so with the help of the SomethingAwful SHSC community, I’ve installed and played with a number of media players over the last couple of days.
Songbird: Perhaps the most full-featured iTunes clone that I’ve ever seen, this app has some serious promise as an iTunes replacement. Memory usage is similar to that of iTunes, but is a little more stable, and doesn’t seem to leak so bad. The interface is fully customizable and skinnable, and it imported my iTunes library in about 5 minutes. Unfortunately, it lacks an equalizer, CD burning or ripping, and has the same poor smart playlist support that iTunes does. I get the feeling that I haven’t even scratched the surface of this players’ feature set, not to mention the hundreds of plugins that you can add from inside the program. Built on the the same XLU framework as Firefox, it’s generally stable, has a tabbed interface, an in-app web browser, and links to SHOUTcast, lastFM, and a ton of other web services. While it’s not quite there yet, I’ll be keeping a close eye on future releases.
Zune: Over the summer, I participated in a Zune marketing program and received a free device in return for reviewing it and the player software. My full reviews can be found over at the Bus Error weblog if you’re interested. The basic story goes like this: Track ratings are either “i like it,” “i hate it,” or “not rated,” which doesn’t provide enough resolution for a large library. The interface is pretty and an interesting departure from iTunes’, but the Zune Marketplace and Social are seriously hampered at best in Canada, which is a shame because I’d probably buy a Zune pass and use this thing to discover new music otherwise. Lastly, when filling the Zune from a smart playlist, there’s no way to limit the number of songs in the playlist by size, so if your library is larger than your Zune capacity, it’s a guessing game. The popular iTunes to Zune playlist converter utility (available in the sidebar) was a project of mine written to address the fact that the Zune is an amazing device with crappy software behind it. Unfortunately iTunes is way ahead of the Zune Jukebox, even though Zune brings some great new ideas to the table.
foobar2000: For what it does, this program is simply amazing. It plays music with my entire library loaded in just under 35MB of memory, and has more customizable features than you could reasonably count. Unfortunately, it lacks ratings and smart playlists (at least i think so – there are a bunch of playlist options that I just don’t understand), and is about as easy to use as reading a novel printed in binary is to read. This app does everything, but for anything more than playing specific songs, albums, or artists, it requires a bunch of reading to learn about. I’m sure that if I spent a week or two learning the ins and outs, I could get the hang of it, but this player is most certainly not for the consumer marketplace – it’s for people who don’t mind taking time to configure it properly and don’t want high level abstraction from the file system.
Media Monkey: One of the only Media Players that I’ve heard of that still offers a paid option. I suppose when you don’t have a store and you’re not open sourced, you have to make money somehow. The free version has a good feature set, and a lifetime license is worth $20, considering the extra features that are enabled with it. On first launch, it took about 10 minutes to index my music folder, detected that I had iTunes installed, and imported all of the library data from it. Unfortunately, this last step took forever, although I could listen to all of my music in the mean time – it just isn’t all tagged properly. At first glance, this appears to be the best jukebox software ever written. It sorts by just about any criteria you could wish for without making playlists, automatically pulls track information from amazon or wikipedia, has podcast, SHOUTcast, and ICEcast, and a web browser built in, and has a direct link to purchase any track in your library from the Amazon store. As of yet, this is by far the most impressive media player that I’ve ever encountered. I will be purchasing the full license and temporarily adopting it as my main jukebox. Expect to hear more about this app in the future.
So there you have it. As per usual, Apple is shiny and simple but doesn’t necessarily include every feature that one could want. Microsoft brings a strong contender to the table, but fails to pull ahead in the race, likely until they throw a few more billion dollars at it. The open sourced Songbird looks promising, but as with most open sourced projects, will need to hit version two before it’s viable for the mass marketplace, and foobar2000, while an example of impressive programming, is so stripped down and customizable that it would confound the average user. Kind of like Linux. Ok, that wasn’t really a fair jab. For now, I’m going to be playing with Media Monkey, and I’ll share my experiences when I’ve thoroughly explored its feature set.
Which player is best for you? If you’re not a computer enthusiast, and don’t feel like paying $20 for a media player, I’d go with iTunes. It’s simple, intuitive, and provides every feature that the average user expects. If you don’t like rating your music, or have a collection of less than 100 songs, check out the Zune player – it’s a neat twist on iTunes, even if it’s not that great for large collections yet. Songbird is most certainly an app to keep in mind, and will probably become a possible iTunes killer as it approaches maturity. Meanwhile, if you want an eye opening experience that will show you how a media player ought to work, look into Media Monkey. It really is cool.
Cheers,
Jon
Edit: As pointed out by my colleague Jake Billo in the comments of this post, some of what I wrote about iTunes in this post was 100% made up and totally incorrect. Not that I was intending to lie about it, but some of my thoughts regarding iTunes were misconceptions. For the record, if you haven’t yet challenged your membership to the cult of Steve, you should still try to do so – there are alternatives out there. And yes, the grass is greener on the other side.
This week on SlightlySauced, our regular crew is joined by special guest Greg Lehman to talk about the impact of the lowly text message on the inhabitants of countries with less developed infrastructures. Listen in to find out how the technology that we take for granted is making a positive impact on those without access to the utilities of communication tha […]
Jon
The primary contributor to and maintainer of the site
Steph
My girlfriend, who sometimes posts her writings
Downloads
Charles Darwin
An essay that I wrote about Charles Darwin’s contributions to Science and Society for a history class at WLU
DRM Essay
An essay that I wrote for an Ethics class I took at Laurier that examines DRM, the USA DMCA, and the failures of both as security against piracy
iTunes Playlist Exporter
Exports all of the songs in any iTunes playlist file to any location on your computer. Originally written to load a blackberry or other mp3 player with music.
MAX 3D Engine
A not quite finished 3D engine written in C++ and OpenGL for my CP411 computer graphics course.
Ted Rogers
A paper that I wrote about Ted Rogers’ personal and business pursuits for a history class at WLU
The Battle of the Atlantic
An essay exploring the lessons learned by both sides during the Battle of the Atlantic in WWII. The essay explores the military and industrial capabilities of the combatants, the technology behind the Enigma and Allied code breaking efforts at Bletchley P
Tile-Based Map Editor
Written in VB for my top-down XNA rpg, allows easy creation of 2D tile-based maps, and exports to both a PNG pallete and an XML map description. Use it or modify it as you see fit.
Bus Error
Jake Billo’s excellent weblog, always good for a laugh or some handy info.
Matthew Good Online
The excellent (although sometimes jaded) blog of Canadian musician Matthew Good.
MusikPolice @ Last.fm
My profile over at Last.fm, one of the few social networking sites that I use.
The Linux Experiment
Seven Windows users with varying levels of Linux experience attempt to run it various distributions on their primary computers for four months. Hilarity ensues.
TylerBurton.ca
The blog of fellow computer enthusiast Tyler Burton, who uses it primarily as a showcase of software he’s written.
