jump to navigation

InterIMAP Progress Update #4 April 29, 2008

Posted by atmospherian in InterIMAP.
Tags: , ,

Here is a break down of this weeks updates to the library:

Configuration Editor (Window app)
I created a simple Windows application that can be used to create and edit configuration files for the InterIMAP library. This should make it much easier to create and manage configuration files.

Local Data Cache Update
So the local data cache has been coming along pretty well. I can now successfully save and load data from the cache and use it to populate the object model upon startup. This is great, however there is much left to be done. For example, there is no synchronization, so if something has changed on the server (folders added/deleted, messages added/deleted), the entire cache would have to be re-generate and saved again. The next step is to design a way to check for changes on the server and changes to the local cache and decide which changes take precedence. Should the local changes be applied first, and then pull the new state of affairs from the server? or should the server be more authoritarian and overwrite the changes made locally when in offline mode?

I implemented a few message management functions on the folder level that allow you to copy, move and delete a message. One problem i ran into though, if the cache is not updated after performing these operations, when i re-ran my test app, messages were loaded that no longer exist on the server and should also not exist in the cache.

Right now the SaveCache method forces every message to be downloaded from the server before everything is saved. But it should behave differently if there is already data from the cache loaded. It shouldnt need to re-download messages that are already stored. Part of the problem i see is that due to the dynamic data loading, most messages by default dont have the content loaded unless the message content has already been accessed elsewhere in the app before the cache is saved.

I will continue to research this and blog more about it as development continues

Bugs Fixed
I Received an email from a developer who was trying to use the SSL functionality to connect to Gmail and got an exception when the system tried to examine the ‘[Gmail]’ folder. The problem was this folder has the \Noselect flag specified which the system wasn’t told to look for and skip that folder if it was found. This has been fixed so the folder processing method will now skip folders that are flagged as not selectable (this applies to EXAMINE as well).

Once that bug was solved, another became readily apparent. After some frustrating debugging and searching i discovered what was causing the ReceiveBuffer method to hang when trying to receive the data from the message body using the SSL connection. I was trying to read from the SslStream object directly instead of the StreamReader object, which both have a Read() method, although slightly different. This has been corrected so you should not have any problems reading data from an SSL connection now.


InterIMAP Progress Update #3 April 28, 2008

Posted by atmospherian in InterIMAP.
Tags: , ,
add a comment

So last week i checked in some pretty important updates that i wanted to go over in a little more detail.

Configuration Serialization
I felt it would be useful to be able to store the configuration settings in an external file that could be read from multiple applications using the library. To facilitate this i created a command line app that walks the user through the creation of the settings file. A settings file can also be created from inside any application by calling the SaveConfig method on an IMAPConfig instance. Right now the config data is being serialized in binary, but i will probably change that to XML so that the config can be editted externally.

In the interest of security i will be researching different encryption methods to encrypt the passwords that are stored in the external file. Doing this has a few pros and cons: The Pro is increased security from people opening the file and seeing what the password to the account is. The con is this method will prevent the config file in its entirety from being editable externally. All the other options can be changed, but trying to change the encrypted password would not work. But i suppose that the need to modify the file manually would be pretty minimal considering there is a quick and easy way to re-generate it. I might build a Windows app to create and edit config files to make it that much easier.

Local Data Cache
I also started to implement the caching of folder and message data to the local filesystem. For this to be useful, all the message and folder information must first be downloaded. Depending on the account this is run on, it can take a considerable amount of time. The account that i am currently using for testing has over 10,000 messages and over 100 folders, so using this account to test the cache might not be the best option so i plan to create a fake account and add some folders and messages to simulate how the cache would work.

In the configuration there is a setting for the name of the cache file, which if specified, can be loaded by the client after logon and used to populate the object model with all the previously downloaded messages. A method would then have to be devised to check each folder for new and deleted messages and update the object model accordingly. As far as i can tell the only flag on a message that can change is whether its been read or not.

I’m looking into whether the library should support a full “offline mode” where if there is a cache available, the developer can specify that no connections to the server should be made, and that only the data stored in the cache would be available. Trying to decide if this should be a library feature, or if its something that should be left to developers to implement in their client code. Certainly including this in the library would save a lot of time for developers using it and increase the utility of the system.

InterIMAP Progress Update #2 April 21, 2008

Posted by atmospherian in InterIMAP.
Tags: , , , ,
add a comment

Well i just got back from vacation where i took a break from all of my development projects. But now i am ready to get back up to speed with InterIMAP.

Dynamic Message Data
Over the weekend i implemented the dynamic message data architecture where the data for a message won’t be downloaded from the server until it is requested. Once any of the header fields are accessed, all the header data for that message is loaded into the object. If any of the data fields are accessed, then a seperate request is sent to get the body and attachments.

This is done to improve the speed at which the client can startup and run without incurring the overhead of loading all the message data for a given folder all at once.

One might say, well what if i need to iterate over all the messages in a folder looking for certain information? wouldn’t that cause all the message data to be loaded anyway? Technically, thats true, but i have an idea to implement search methods in the client wrapper that would allow you to search for text in various parts of the message, on the server directly and return a list of the messages that match that search. Once you have that list you would only be loading the data for a smaller set of messages.

I also have an idea for (de)serializing folders to XML and binary formats to create local caches of messages much like Outlook does for Exchange. The idea would include a mechanism in the IMAPConfig class to specify the filename(s) of the cache files to load. Once the data is loaded from the cache, a synchronization could be run which would check each folder in the cache and on the server looking for new messages and adding those messages to the object model. Once in the object model, the folders would need to be reserialized.

This idea will hopefully provide another means of increasing the speed at which the client can interact with the server and reduce the number of requests that have to be sent back and forth. Please let me know what you think of this idea and whether it would be useful, or a waste of time.

I am also toying with the idea of including support for SMTP to provide a seamless method to forward messages to other addresses, and send new messages. More research will be needed before i make a final decision as to include this or not, stay tuned for further updates.

InterIMAP Progress Update #1 April 11, 2008

Posted by atmospherian in Projects.
add a comment

Work on the library has been moving along quite nicely. I’ve built a new Visual Studio solution and project structure to better organize the classes and i have finished the basic object model.

The next step is to write a wrapper class that will be the main API users of the library will use to access the underlying functionality. The main goal of the API is to make connecting to an IMAP server and managing the messages stored there as easy and a quickly as possible.

One method that i plan on employing to increase the efficiency is to only fetch the message headers and message bodies when they are requested. This will be handled automatically whenver a property is accessed on either the folder or message object. For instance the first the developer accesses the Messages collection of a folder, first that folder will be examined, and the message headers will be pulled down and stored in the collection. This will give access to the bulk of the messages information. If at that point the content or attachments collection are accessed, then the full body of the message will be downloaded.

Hopefully this method will help to reduce calls to the server for unneeded information and reduce that amount of bandwidth that will be needed for the IMAP client to do its job.

I welcome any comments or criticisms on what i am planning to do. All suggestions will be given serious consideration if the community decides a different way is a better approach.

InterIMAP April 9, 2008

Posted by atmospherian in C#, Projects.
Tags: , , , ,
add a comment

It’s official. I’ve received the blessing of Rohit Joshi to continue his work on his C# IMAP client library he posted over at codeproject.

I’ve created a project at codeplex to host the library and its code. You can find it here: http://www.codeplex.com/InterIMAP

Some major features that i will be adding to the library include:

  • Complete object model for folders, messages, content (including attachments)
  • Support for SSL connections (Complete)
  • Support for saving file attachments to the file system, or any other Stream
  • Support for getting the list of folders on the server and getting the UIDs of the messages in each folder
  • Being able to do all of the above with a few API calls as possible
  • The ability to send a raw IMAP command to the server and retrieve the response. Useful for customizing the behavior of the library.

I am really excited about this opportunity to contribute to the open-source community. Hopefully people will find this library useful and maybe help contribute to it and make better.

Welcome April 8, 2008

Posted by atmospherian in General.
Tags: , , , , , , , , ,
add a comment

Hey There!

This is my new blog where i will be posting information, code and other cool stuff that i come across during my software engineering escapades.

I have a few projects in the works right now which i will blog about in the coming days as things slowly materialize.

My main areas of focus are C#, WPF/Silverlight, LINQ, SQL Server, J2EE, MySQL, ASP.NET and AJAX. One of my projects is a very ambitious cross-browser, cross-platform personal data aggregator and indexer. It will be comprised of a desktop application for Windows, Mac OS X and Linux, and Web site. One of the main challenges of this project will be figuring out the best language/platform that will offer the most code reusability but not limit the functionality or asthetics.

I am planning to use WPF for the desktop app on Windows, and i’ve recently discovered a XAML implementation for Java that integrates into the Eclipse IDE that works on Mac OS X and Linux, which looks like it will help with porting the UI to those platforms even if it means having to re-write the logic behind it…

I guess thats enough for a first post. As my thoughts on these projects form some kind of jelly like consistency, i will post more information, as well any other cool stuff i find along the way.