jump to navigation

InterIMAP Progress Update #7 May 22, 2008

Posted by atmospherian in InterIMAP.

I am very excited to let you guys know that the re-engineering of the message processing architecture is currently stable and working. My new system for processing message content has not only reduced the number of send/receives from the server to a maximum of 3, but has reduced the code of IMAP.cs by roughly 50-60% making it much cleaner and easier to read and understand.

The original way of processing the message content, which was written by the original author of the code, Rohit Joshi, worked well, except for the fact that it choked on many messages from Gmail, which as i understand is one of the main services that this library is being used with (Exchange is the other). Modifying his code to work with gmail proved nearly impossible due to the extreme variations that can occur in body structure. The process involved first getting the BODYSTRUCTURE of the message, parsing that and then getting each individual BODY[x(.x.x)] section. While this is a valid solution, it is very difficult to write something that can parse the BODYSTRUCTURE in its seemingly infinite variations.

The solution is to pull the full content of the message directly using the BODY[] parameter. The result of this command is every section of the message being return in sequence, usually demarcated with a ‘boundary’ value (except in the case of messages with a single text/plain content type). After this boundary, is the header meta data for that section, which can include the content-type, transfer encoding, disposition, etc. making it easy to parse this header data and properly construct IMAPMessageContent objects.

What does this mean for you? it means that there is now a complete, accurate, and efficient process in place for retrieving the message content from both gmail and exchange.

So whats next? Well, during testing, it became clear to me that for large accounts, having all of the message attachments stored in memory could require a significant amount system resources, much more than an email system should need. So with that in mind i am starting to think about ways to store attachments outside of the main cache file, so that when the cache is loaded, the attachments arent loaded until they are requested.


InterIMAP Progress Update #6 May 15, 2008

Posted by atmospherian in InterIMAP.
add a comment

Major updates in the works this week. After some intial testing with GMail, several show-stopping bugs were discovered in the body structure processing code. Modfying the existing code wasn’t possible as the differences were too severe. Instead i opted to re-write the entire message parsing system to be more flexible and to directly interact with the object model, as opposed to generating XML data first and then using a DataSet to read in that XML which was a cheap hack i employed to get to the message data.

I am not quite ready to upload the newest code as i want to perform some more testing with gmail and exchange before i feel it is ready for a wider audience. There is one bug in particular that is show-stopping with gmail that i am trying to address. Once i get that cleared up we should be in good shape.

Stay tuned for more updates as testing on the re-write continues…

InterIMAP Progress Update #5 May 7, 2008

Posted by atmospherian in InterIMAP.

New updates coming this week:

Simple and Advanced Searching
I have added a new search system to the library that should make searching within a folder very simple. There are two new objects in the namespace, IMAPSearchQuery and IMAPSearchResult. The SearchQuery object contains many properties that represent the various search options that can be specified.

The actual Search method is defined in IMAPFolder and takes a IMAPSearchQuery instance as a parameter. The search method returns a IMAPSearchResult object that contains a reference to the query that was used, the folder that was searched and a list of IMAPMessage objects that are the results.

To simplify the search process, several static methods have been defined in the IMAPSearchQuery class that take a single argument, and return an IMAPSearchQuery object, which can be plugged directly into the search method like this:

IMAPFolder f = client.Folders[“INBOX”];
IMAPSearchResult sResult = f.Search(IMAPSearchQuery.QuickSearchSubject(“IMAP”));

What the above code basically does is first gets the folder named INBOX, and then calls the Search method on that folder using the one of the quick search static functions to automatically create an IMAPSearchQuery instance to use for the search. The results of the search are stored in sResult and the messages that match the search query can be found in sResult.Messages.

For more advanced searching you can create a IMAPSearchQuery object manually, and specify the terms that you wish to search for. You can specify values for any of the properties that are defined in that class and each will be taken into account during the search.

Currently this system will work only for including all the messages that match the query. There is no support for ‘OR’ or ‘NOT’ style parameters as of yet. The plan i have for this is to add special List<IMAPSearchQuery> properties to the query object, one for ‘NOT’ and one for ‘OR’ where new SearchQueries can be added and processed and included into the main query. I have to experiment with this option and see if it will work well and be intuitive and easy to use.

Local Data Cache
I’ve spent a great deal of time this week working on the local cache system. I’m happy to say that it’s nearly complete. Here is how it currently works:

First, to enable the local cache the developer needs to specify a cache file and cache format in the configuration. This can either be done at runtime, or using the configuration generators.

In the constructor for the IMAPClient class, the configuration is checked to see if a file is defined. if it is, a flag is set, and if autologon is also specified, the system logs in.

Once logged in, the system checks if the specified cache file already exists. If it does, it loads the cache into the object model, and then (if not in offline mode) the system syncs the cache with server (a config option has been added to set the system to auto sync or not). Updates are one-way with the server being the authority. (If the system is running in offline mode, any functions that would change the folder contents or structure are disabled). If the file does not exist, the system then begins the process of building a new cache file. Please be warned that if you are connecting to a large account with many folders and messages this process can definitely take some time (several hours in the worst cases, depending on the location of the server relative to your location, ie. if the server is on a local LAN it will be faster than pulling from the internet).

Once the cache is built, all the data will already be in the object model so no prep work is required to start accessing the messages.

When working in online mode, all of the message and folder copy/move/delete methods will also update the cache when their process completes successfully. This update should not force any special server commands to be run, it should simply serialize the current state of the object model.

InterIMAP Feature Overview May 1, 2008

Posted by atmospherian in InterIMAP.
1 comment so far

Note: this post will be updated as the current state of the feature set changes.

Current Features

  • Dynamically loaded message data
  • Caching of message data including attachments to filesystem
  • Supports SSL connections
  • Ability to run in offline mode when using caching system
  • Ability to create and delete folders and copy/move/delete messages
  • Support external configuration files for all options
  • Includes Windows and command-line tools for creating and managing configuration files
  • Provides advanced search system that works in both online and offline modes
  • Ability to save attachments to local filesystem

Planned Features

  • Nested search queries using NOT and OR operators
  • Include simple e-mail client to demonstrate use of the library