csammisrun

A rare situation

Archive for the ‘Development’ Category

.NET 3.0 functionality in .NET 2.0: a sorting, filtering, bindable list – Part Two

with one comment

The thrilling conclusion to Part One!

When we left our hero, the BindingListCollection, we had a nicely serviceable collection that could be bound, sorted, filtered, and was .NET 2.0 compliant. New items added into the list were sorted and filtered correctly, and life was wonderful…but it was not perfect yet.

Ch-ch-ch-ch-changes (don’t want to be a static list)

If you have a contact list with more than zero people on it, chances are there are going to be events that alter the state of the list in some way. Contacts change status, occasionally their names change, the number of contacts in a group can change, and so on. A contact list is a highly mutable structure, and we want the data structure containing it to be able to respond dynamically.

I mentioned in part one that the BindingListCollection implements INotifyPropertyChanged, so that anything that binds to it can be made aware when a property on the collection changes. I also mentioned, in an off-hand way, that all good databound classes implement that interface. The core data structures that make up the shaim contact list – ContactGroup and MetaContact – implement INotifyPropertyChanged. Maybe the BindingListCollection should take advantage of this! We’re going to require that anything contained in a BindingListCollection implements INotifyPropertyChanged by putting a restriction on the generic type used by the container.

public class BindingListCollection : BindingList, INotifyPropertyChanged where T : class, INotifyPropertyChanged

Now that we know for a fact that all items being inserted into the list will have the PropertyChanged event, we can update InsertItem to attach event handlers when a new item is added:

...
    // Insert the item into the list
    if (index != -1)
    {
        if (index > base.Count)
        {
            index = base.Count;
        }
        base.InsertItem(index, item);
    }

    INotifyPropertyChanged propitem = item as INotifyPropertyChanged;
    if (propitem != null)
    {
        propitem.PropertyChanged -= item_PropertyChanged; // Don't attach it twice
        propitem.PropertyChanged += item_PropertyChanged;
    }

    // Fire the AddingNew event
    OnAddingNew(new AddingNewEventArgs(item));
...

The event handler is removed in RemoveItem to avoid stray references that prevent garbage collection.

Now let’s think about the event handler method, item_PropertyChanged. The PropertyChanged event has only one parameter: the name of the property that changed on the item that raised the event. We don’t want to muck with the list on every property change, though – this is where the BindingListCollection starts to get specialized for shaim. With a minimal amount of effort, it could be made to work in generic applications.

shaim supports sorting/filtering by a metacontact’s display name (alphabetic sort), a metacontact’s status (status sort, or hide offline contacts), a group’s display name (alphabetic sort), or a group’s count (hide empty groups). With that in mind, we can listen for only those specific properties, and fiddle with the list contents only if one of those changes.

void item_PropertyChanged(object sender, PropertyChangedEventArgs e)
{
    T item = sender as T;

    // If the list isn't sorted or filtered, no rearrangement is necessary
    if (!isSorted && !isFiltered)
    {
        return;
    }

    // Lists of groups can be sorted/filtered by display name and online count,
    // lists of metacontacts by display name and status
    if (e.PropertyName == "DisplayName" || e.PropertyName == "Status" || e.PropertyName.EndsWith("Count"))
    {
        MoveWithinList(item);
    }
}

MoveWithinList: thanks for being terrible, WPF TreeView

Before I start raving incoherently about a bug that took for-freakin’-ever to track down and squash, I shall present and explain the code listing for MoveWithinList and its support methods:

/// <summary>
/// A constant value returned by FindInCurrentList
/// when the requested item is not in the list
/// </summary>
private const int ITEM_NOT_IN_LIST = -1;

/// <summary>
/// Move an item within the list by its current properties
/// </summary>
private void MoveWithinList(T item)
{
    bool fireItemsChangedEvent = false;
    int originalPosition, newPosition;

    // Remove the item at its current position
    originalPosition = FindInCurrentList(item);
    if (originalPosition != ITEM_NOT_IN_LIST)
    {
        Items.RemoveAt(originalPosition);
        fireItemsChangedEvent = true;
    }

    // Reinsert it at its new position, maybe
    newPosition = PlaceByCurrentComparison(item);
    if (newPosition != ITEM_NOT_IN_LIST)
    {
        Items.Insert(newPosition, item);
        fireItemsChangedEvent = true;
    }

    if (fireItemsChangedEvent)
    {
        // Figure out which list changed event to fire, given the change
        // in the active list.

        if (IsValidPosition(originalPosition) && IsValidPosition(newPosition)
            && originalPosition != newPosition)
        {
            // An item was valid, it's still valid but it moved, fire a...something
            // [trombone making that wah-wah-waaaaaah sound]
        }
        else if (IsValidPosition(originalPosition) && !IsValidPosition(newPosition))
        {
            // An item was valid, now it's not, fire a Delete event
            OnListChanged(new ListChangedEventArgs(ListChangedType.ItemDeleted, originalPosition));
        }
        else if (!IsValidPosition(originalPosition) && IsValidPosition(newPosition))
        {
            // An item was invalid, now it is, fire an Add event
            OnListChanged(new ListChangedEventArgs(ListChangedType.ItemAdded, newPosition));
        }

        if (PropertyChanged != null)
        {
            PropertyChanged(this, new PropertyChangedEventArgs("Items"));
        }
    }
}

/// <summary>
/// Returns a value indicating whether the given position is valid for the list
/// </summary>
private static bool IsValidPosition(int position)
{
    return position > ITEM_NOT_IN_LIST;
}

/// <summary>
/// Returns the index of an item in the currently displayed list
/// </summary>
private int FindInCurrentList(T item)
{
    return Items.IndexOf(item);
}

The logic here is pretty simple. When an item’s property changes, one of four things will happen: it may be moved by the current sort, it may become visible if it was previously filtered, it may become hidden by the current filter, or it might do nothing. MoveWithinList locates where the item is currently, where it’s going (if anywhere), does the rearranging, and fires a list changed event to tell binders what just happened. This completes the mimicry of INotifyCollectionChanged.

Almost completes it, that is. Note the comment in the case where the item moves in the currently visible list. This is not as intuitive as one might hope. If you look up the MSDN documentation for the ListChangedType enumeration, you’ll see ItemAdded and ItemDeleted – both of which are already used by MoveWithinList – and also ItemMoved. How simple! In the mystery event area, just put:

OnListChanged(new ListChangedEventArgs(ListChangedType.Move, newPosition, originalPosition));

…and just wait for the horrific exceptions to roll in!

Unexpected collection change action 'Move'.
  at System.Windows.Controls.TreeViewItem.OnItemsChanged(NotifyCollectionChangedEventArgs e)
  at System.Windows.Controls.ItemsControl.OnItemCollectionChanged(
     Object sender, NotifyCollectionChangedEventArgs e)

Oh boy! The WPF TreeView apparently doesn’t support moving items under a single TreeViewItem. I wonder why this is? Let’s visit our good friend, Reflector:

protected override void OnItemsChanged(NotifyCollectionChangedEventArgs e)
{
    switch (e.Action)
    {
        case NotifyCollectionChangedAction.Add:
            return;

        case NotifyCollectionChangedAction.Remove:
        case NotifyCollectionChangedAction.Reset:
            if (this.ContainsSelection)
            {
                TreeView parentTreeView = this.ParentTreeView;
                if ((parentTreeView == null) || parentTreeView.IsSelectedContainerHookedUp)
                {
                    return;
                }
                this.ContainsSelection = false;
                this.Select(true);
            }
            return;
    }
    throw new NotSupportedException(SR.Get(SRID.UnexpectedCollectionChangeAction, new object[] { e.Action }));
}

Going back up the call stack of the exception, I see that ListChangedType.ItemAdded gets changed to NotifyCollectionChangedAction.Add, ItemDeleted gets changed to Remove, and Reset stays the same. ItemMoved, which gets changed to NotifyCollectionChangedAction.Move, is simply not supported by WPF’s TreeView. C’est la vie, I suppose, let’s just go with what is supported. Instead of the ItemMoved ListChanged event, do a reset instead:

if (IsValidPosition(originalPosition) && IsValidPosition(newPosition)
    && originalPosition != newPosition)
{
    OnListChanged(new ListChangedEventArgs(ListChangedType.Reset, -1)); // <-- f-f-f-frickin hacks!
}

Fortunately this happens less than you might think. Most of the time, a change event happens slow enough that it triggers an add and delete instead of a full reset.

Getting at the real stuff

Consumers of the BindingListCollection may still want to know about what's underneath the filtering and sorting, so we'll expose a couple properties to let them get at that data (but in a read-only sense).

/// <summary>
/// Return the total number of items in the list, visible or not
/// </summary>
public int TotalCount
{
    get
    {
        if (isSorted || isFiltered)
        {
            return unsortedList.Count;
        }
        else
        {
            return Items.Count;
        }
    }
}

/// <summary>
/// Gets a read-only view of the underlying list without filtration or sorting
/// </summary>
public ReadOnlyCollection<T> UnderlyingList
{
    get
    {
        if (isFiltered || isSorted)
        {
            return new ReadOnlyCollection<T>(unsortedList);
        }
        return new ReadOnlyCollection<T>(Items);

    }
}

To finish up, we still have to advertise that the BindingListCollection can do these fun things, so binders can take advantage of it. This is as simple as overriding the various "Supports____" properties of the BindingList and returning true.

That's it! For reals this time!

Oh wait the basil-lime sorbet recipe

Yeah it sucked. Too much simple syrup, not enough lime juice. The basil was good though.

THE END.

Written by Chris

July 17th, 2008 at 9:53 am

.NET 3.0 functionality in .NET 2.0: a sorting, filtering, bindable list

with 3 comments

The third in a series of write-ups on some of the odder parts of shaim.

  1. WPF versus IM typing notifications
  2. WPF windows, in my Winforms application?

As I described in the last write-up, shaim is organized around a central core. The core provides services such as preference loading/saving, loading plugins and dispatching inter-plugin events, and a factory for proxied sockets. It also holds the most central data structure to any instant messaging application: the contact list. This post is going to talk about creating a data structure that replicates the functionality of a .NET 3.0 INotifyCollectionChanged collection in .NET 2.0-compatible code. Why would you want to do that in the first place? All shall be made clear…

Representing and displaying the contact list

For most IM systems and for the purposes of this discussion, a contact list is hierarchical. Stemming from the root of the list are groups, each of which can contain one or more contacts

Contact List
|
+-- Group
|   |
|   +-- Contact
|   +-- Contact
|
+-- Group
    |
    +-- Contact

Hierarchical data of this nature is naturally represented using a tree view, and shaim’s UI plugin is no exception. It uses the WPF TreeView control, which is capable of binding to a list-of-lists and displaying the data in a pleasing fashion. If you bind to a collection that implements the INotifyCollectionChanged interface, the TreeView will even do such kind things as update the view automatically when the bound list changes.

A challenge!

One of the driving goals of the project is to maintain a core that can be compiled on Mono, which means the core uses absolutely no .NET 3.0 components or data structures. The modularity of the plugin design still lets the framework utilize plugins that are based on .NET 3.0 technology, but as long as the core maintains its purity, it’s all good in our hood. Tra la la, happy days…oh shit, INotifyCollectionChanged is a .NET 3.0 interface. The contact list can’t use it, the TreeView won’t update itself properly, chaos ensues, and I’m lying in a ditch somewhere having been beaten to death by hired goons.

Well that’s alright, that’s okay, no one ever said life was simple. We’ll just implement a collection with similar – and compatible – behavior using .NET 2.0. In fact, since people like to do silly things like organize their data alphabetically (pfffft), let’s add sorting logic to it. And while we’re being all nutty, let’s add a filter so users can remove items that meet certain criteria, then people can do such as not display empty groups or offline contacts. Above all, it has to be painless from the UI’s point of view: we don’t want any of this logic floating up into the UI, where it would have to be replicated by any new UI plugin.

So this collection has to be:

  • .NET 2.0 compatible
  • Bindable by WPF and Winforms
  • Sortable
  • Filterable
  • Hell, let’s make it generic while we’re at it

We shall call it: BindingListCollection!

The basics of the BindingListCollection

First things first, we have to find .NET 2.0 structures and interfaces that we can start with that’ll let us fake the funk appropriately.

Like all good classes that are bound in some way, the BindingListCollection will implement INotifyPropertyChanged. This interface is similar to the verboten INotifyCollectionChanged, in that it notifiers consumers of a change in the implementing object, but instead of a change on the contents of a collection the INotifyPropertyChanged interface notifies of changes in a single property. Perhaps we can fool the TreeView into refreshing its view of the collection by notifying of a property change on the Items property! In fact we can, this does work, but it’s pretty horrific. Anything binding to a specific property completely refreshes when a property change happens, which gets expensive and slow when you’re talking about a list of any sort of appreciable size. The INotifyCollectionChanged interface gets around this by raising an event that specifies exactly what action was taken (adding or removing an item, for instance) and where in the list it happened, so the entire thing isn’t munged by what should be a quick change.

Is there an interface or a class in .NET 2.0 that is clever like the INotifyCollectionChanged interface? Turns out that there is: BindingList. It fits all of our requirements: it’s .NET 2.0, supports binding, has support for sorting and filtering, and is generic. So begins our new type, BindingListCollection.

public class BindingListCollection<T> : BindingList<T>, INotifyPropertyChanged

Now whenever we manipulate the collection (by adding, removing, filtering, or sorting items), raising the ListChanged event will cause whatever’s binding to it to update correctly and specifically, just like INotifyCollectionChanged.

Filtering and sorting: adding support for all this ballyhoo

The BindingList provides support for filtering and sorting, but leaves it to you to implement the routines to do it. Let’s make this super-extensible, and create methods to filter and sort using generic delegates for the purpose. Consumers of the BindingListCollection can create custom methods to sort and filter, then pass it in and let the BindingListCollection do the rest.

private bool isSorted;
private bool isFiltered;
/// <summary>
/// The sorting comparison currently in use
/// </summary>
private Comparison currentComparison;
/// <summary>
/// The filtering predicate currently in use
/// </summary>
private Predicate currentFilter;

/// <summary>
/// Sorts the list by the specified property in the specified direction
/// </summary>
public void Sort(Comparison sortCriteria)
{
    isSorted = true;
    currentComparison = sortCriteria;
    ApplyListAlterations();
}
/// <summary>
/// Filters the list contents by the specified predicate
/// </summary>
public void Filter(Predicate filterCriteria)
{
    isFiltered = true;
    currentFilter = filterCriteria;
    ApplyListAlterations();
}
/// <summary>
/// Applies the current filter and sort to the list
/// </summary>
private void ApplyListAlterations()
{
    List listref = Items as List;

    Items.Clear();
    listref.AddRange(unsortedList); // An original copy of all items - explained below

    if (isFiltered)
    {
        // Perform filtering on the current item list
        for (int i = 0; i < Count; i++)
        {
            if (!currentFilter(Items[i]))
            {
                Items.RemoveAt(i--);
            }
        }
    }

    if (isSorted)
    {
        // Sort the current item list
        listref.Sort(currentComparison);
    }

    // Tell all bindings to reset completely
    OnListChanged(new ListChangedEventArgs(ListChangedType.Reset, -1));
}

That was it! We now have a filterable sortable bindable generic collection...but it sucks! If you add a new item to the BindingListCollection, it doesn't get sorted or filtered. Right now a consumer has to call Sort and Filter again every time an item is added, which causes a complete reprocessing and binding reset, and that is e-x-p-e-n-s-i-v-e. We have to be clever to achieve awesomeness.

The essence of filtering is removing items from the list, but we have to save those items somehow; if we didn't, every time the filter was changed, or removed, or the item changed so that the filter no longer applies, the item would have to be re-added. The essence of sorting is changing the order in which items are displayed, but we want to preserve the original order. Both of these can be accomplished by maintaining a backing list - a list of all the items, exactly as they're added, that can be used to restore order when the sort or filter changes. We've already seen this in the BindingListCollection: the 'unsortedList' mentioned in ApplyListAlterations().

/// <summary>
/// The underlying list of manually inserted items
/// </summary>
private List<T> unsortedList = new List<T>();
/// <summary>
/// The index at which the item was originally inserted by the list controller
/// </summary>
private readonly Dictionary<T, int> insertIndicies;

The backing list must be maintained whenever items are added or removed from the BindingListCollection, so the InsertItem and RemoveItem methods are overriden to add to both the backing list and the main list. InsertItem is not terribly complicated:

/// <summary>
/// Inserts the specified item in the list at the specified index.
/// </summary>
protected override void InsertItem(int index, T item)
{
    // Maintain the backing list
    if (index > unsortedList.Count)
    {
        unsortedList.Add(item);
    }
    else
    {
        unsortedList.Insert(index, item);
    }

    // Add the requested index to the item-index map
    insertIndicies[item] = index;

    // Insert the item into the list
    if (index != -1)
    {
        if (index > base.Count)
        {
            index = base.Count;
        }
        base.InsertItem(index, item);
    }

    if (PropertyChanged != null)
    {
        PropertyChanged(this, new PropertyChangedEventArgs("Items"));
    }

    // Fire the AddingNew event
    OnAddingNew(new AddingNewEventArgs(item));
}

I'm not going to specifically list out RemoveItem here, but it's pretty simple: Remove the requested item from the backing list and insertIndicies, remove it from the base list, and that's that.

Filtering and sorting: what goes where?

Even though every item added to the BindingListCollection ends up in the backing list, it may not end up in the requested insert position if a sort is applied, and it may not end up in the main list at all if a filter is applied. The BindingListCollection has to compute an insertion index that makes sense according to the current sort and filter, and that's done using a new-fangled method, PlaceByCurrentComparison:

/// <summary>
/// Returns the index at which an item would be inserted by the current sort comparison
/// and current filter
/// </summary>
/// <returns>The index at which to insert the item, or -1 if it would not be in the currently filtered list</returns>
private int PlaceByCurrentComparison(T item)
{
    // If the item doesn't match the current filter, don't add it
    if (isFiltered && !currentFilter(item))
    {
        return -1;
    }

    int insertindex = 0;
    if (currentComparison != null)
    {
        // Get the index in the sorted list
        while (insertindex < Count && currentComparison(item, Items[insertindex]) >= 0)
        {
            insertindex++;
        }
    }
    else
    {
        // Check the originally requested insertion index, put it as close as possible
        if (insertIndicies.ContainsKey(item))
        {
            while (insertindex < Count &&
                (insertIndicies.ContainsKey(Items[insertindex]) && insertIndicies[item] >= insertIndicies[Items[insertindex]]))
            {
                insertindex++;
            }
        }
    }

    return insertindex;
}

...and the InsertItem method gets updated:

...
    // Add the requested index to the item-index map
    insertIndicies[item] = index;

    // If the list is sorted, find the correct insertion point for the sorted order
    if (isSorted || isFiltered)
    {
        index = PlaceByCurrentComparison(item);
    }

    // Insert the item into the list
    if (index != -1)
    {
        if (index > base.Count)
        {
            index = base.Count;
        }
        base.InsertItem(index, item);
    }
...

This seems pretty nice now: items are inserted into the list, their requested insert indexes are saved, but they may or may not be displayed sorted or at all, and the BindingListCollection takes care of all of it. The caller sees only this:

BindingListCollection<ContactGroup> groupList = ...;
groupList.Sort(someAlphabeticComparison);
groupList.Add(new ContactGroup("Buddies"));
groupList.Add(new ContactGroup("Amigos"));
groupList.Add(new ContactGroup("Wise-ass jerks"));

I am so done with typing right now

This is getting pretty ridiculously long for a blog post, so I'll finish off the discussion later this week. Still to come: Live updating of the filter and sort as items change, overriding properties to make sure the bound UI object knows what can be expected, and a failed recipe for basil-lime sorbet.

Written by Chris

July 16th, 2008 at 1:38 pm

WPF windows, in my Winforms application?

with one comment

In the last post about some of the odd things that shaim does, I talked about how WPF’s rich text input makes for laggy typing and how we lessened the impact. Today I’ll talk about how one should be grateful that any text input happens in shaim at all, due to the slightly strange way that WPF is used by the application.

Core ‘n’ plugins

The shaim architecture is plugin-based. There’s the core (shaim.exe, a Winforms Application), and several plugins that comprise the bulk of the application’s functionality. Many people don’t realize that the UI – the contact list, conversation windows, all of that noise – is a plugin itself, specifically a WPF Class Library.

When we were designing shaim back in the bad old days, we knew that we were going to use WPF for the UI, so why isn’t the core a WPF application? The answer is compatibility: WPF is only supported on Windows XP SP2 and up. We wanted to build shaim so that developers could make UI plugins with different windowing toolkits so it could run on other platforms. This has actually worked out in practice, as the core is fully .NET 2.0 compatible and there’s at least one community-driven UI plugin in the works (using GTK#).

Two event models

About the only thing that Winforms and WPF have in common is that they’re both Windows GUI toolkits. Winforms follows the general Win32 GUI event model: An Application has a message pump that raises events (mouse click, text typed, window moved, etc.), developers handle events as they are raised from the message pump, the world keeps on spinning. There are niceties to the process, but at its root that’s all there is to it. It’s not altogether difficult to see how Winforms is a shell over the Win32 API.

WPF has a different event model altogether, called routed events. Although the concept of direct events still exist, there are also now tunneling and bubbling events. The gist of it is this: The user types text into a text field, which generates a PreviewKeyDown event from the root element. This event “tunnels” down the visual tree until it gets to the text field, which then generates a KeyDown event, which “bubbles” back up the visual tree until it gets to the root element. At any point along the visual tree, either type of event can be handled.

Where all the keyboard events at?

When shaim’s UI and WPF itself were still very young – we’re talking like January / February 2006 – we noticed that we couldn’t type into any text input fields. Buttons clicked, windows moved, but no keyboard events were fired at all. Attaching event handlers to all the key press events in the visual tree, we found that all the tunneling Preview* events were being raised, but none of the corresponding bubbling events were happening, and no text would appear in the input field.

As much fun as having an IM client that you couldn’t use to send messages seemed, we decided to try hosting a Winforms text field control in one of the UI windows to see what would happen. Lo and behold, it worked just fine. Great, except at the time hosting was slow and rather bloated, not to mention it felt like a cheap hack. What the hell was wrong with WPF text input?

Two! Two Application objects! Ha ha ha!

After a lot of experimenting and talking to a Microsoft engineer (I forget who, sorry), we finally got it diagnosed. It was completely shaim’s fault for being architected as a Winforms application that launched WPF windows, instead of a straight WPF application or even a Winforms application that lauched Winforms windows that contained a Winforms->WPF host. There was no WPF message pump, so WPF-style routed events couldn’t happen correctly. Direct events, like clicking and moving windows, were still fine. The Winforms text field worked because the shaim core hosts a Winforms message pump in its Application object.

Okay, so give the UI plugin a message pump and it’ll be happy…but starting a message pump by calling Application.Run() is a blocking call. If the Winforms Application started a WPF Application, it couldn’t do much else. The stunning workaround: Start the WPF Application on its own sweet time.

void StartWpfApplication()
{
  application = new Application();
  application.Startup += application_Startup;

  AnonInvoker invoker = delegate
  {
    // Some slow operation that we didn't want to block core startup
    application.Run();
  };
  application.Dispatcher.BeginInvoke(DispatcherPriority.Normal,
    invoker);
}

void application_Startup(object sender, StartupEventArgs e)
{
  // signal the core that the UI's ready to roll
}

And that is why shaim users should be damn grateful that they can type in shaim at all

Written by Chris

April 4th, 2008 at 2:53 pm

JabberLib 0.1 released

without comments

I’m pleased to announce that JabberLib, a .NET library for connecting to XMPP (Jabber) servers, has just been released for the very first time on the Internet that you are reading! Right now!

Download the 0.1 release here

Who is this for?

JabberLib, similar to OscarLib and MsnLib, is a component for .NET developers to add Jabber messaging capability to their applications. It is written in C# and is .NET 2.0- (and Mono-) compatible.

Can I use this in my closed-source application that I sell for cash monies?

Sure, go right ahead! JabberLib is released under the MIT License and is freely available to all developers without restriction (or guarantee).

How can I add this to my project?

Quick and easy:

  • Download the 0.1 binary release and extract it to your project directory
  • Right-click on your project in Visual Studio and choose “Add reference…”; go to the “Browse” tab and select JabberLib.dll from wherever you extracted it.
  • See the example code to learn how to instantiate and manipulate a JabberLib Session.

I found a bug / I want to request a feature

Bug reports and feature requests can be logged on the JabberLib project site.

I want to contribute code, kind words, or death threats!

Stop on by #shaim on irc.freenode.net and chat with the developers. Most of us live in the East Coast / Central United States, but someone is always idling in the channel so feel free to leave a message.

Original posting on shaim.net

Written by Chris

March 28th, 2008 at 9:31 am

WPF versus IM typing notifications: how to type a 10 word sentence, get some coffee, and continue watching it render

with 5 comments

Hello and welcome to what I hope will be several write-ups on some of the stranger things that happen in shaim. As the project barrels towards its next significant release and we gain both developers and users, the system as a whole is undergoing something of a critical examination. Today’s topic concerns typing in the conversation window, something that until recently has been an extremely slow experience and every shaim user feedback to date has called out as being a barricade to using it on a day to day basis.

WPF and text input

shaim’s UI is written in Windows Presentation Foundation, which is pretty great for displaying all manner of things in very shiny ways, but has some problems with input and rich content input in particular. shaim, for better or for worse, uses a great deal of rich content input by way of the RichTextBox (RTB) class, extended by shaim to include support for HTML.

“What sort of problems does WPF have with text input?”

In general, it’s slow. In an effort to support everything under the sun, the WPF implementation of the RTB seems to have outgrown itself. Projects like SharpDevelop that require specialized text editing with rich content support tend to roll their own. There’s a noticeable lag between typing a stream of characters and seeing the results appear, and doing anything in an event handler on text input events naturally slows the process a bit more.

shaim and typing notifications

Most IM networks support the concept of typing notifications, small notification packets that indicate when a user is typing to another user. In general, this is a pretty simple concept: send a notification when typing starts, send one when typing stops. If the actual message is received, the client can assume that means typing has stopped (until the next notification is received). shaim’s typing notification support is based around AIM’s very special implementation of the feature, which features not two but three notification types:

  • Typing started: pretty clear
  • Typing paused: there is text in the user input field, but it hasn’t been sent yet
  • Typing stopped: the text field has been cleared

shaim handles the RTB’s TextChanged event to provide support for sending typing notifications. The basic flow is as follows:

In TextChanged event:
  Is text blank?
    Send TypingStopped
  If text is not blank and TypingStarted has not been sent
    Send TypingStarted
    Start a 2 second timer
      When the timer elapses, send TypingPaused unless TypingStopped was sent in the interim

Pretty simple, and it’s hard to boil that down any more with the three-state model. But it was dog slow, even slower than usual, and here’s why:

Is text blank?

It turns out that this is a bit more of a puzzler than you might expect. The RTB doesn’t expose a Text property, unlike a normal TextBox, but that fact itself isn’t unexpected: an RTB can hold markup, images, and all sorts of things that don’t belong in a string. Even so, it would be nice to have the plain text for various reasons. shaim’s RichEditControl fakes the funk by exposing a PlainText property that extracts the text content from the RTB.

So the test for blank text became

if (UserInput.PlainText.Length == 0)

Great, until you realize that this is being called in an event handler that is raised on every text change event – every single keystroke. The text extraction process isn’t cheap, and profiling showed that this caused a great deal of the text entry slowdown.

The next idea was to give the RichEditControl a new property, IsEmpty, that simply compared the RTB’s ContentStart and ContentEnd TextPointers to see if they were equal. If they were equal, the content was zero-length and the RTB was empty.

if (UserInput.IsEmpty)

Though it was indeed faster code-wise, there was still a very perceptible typing lag.

Okay, just forget it

It turns out that accessing any of the RTB properties from an event handler chain that is updating those properties isn’t cheap and was causing what amounted to unacceptable lag. The final solution? Don’t use the editor’s properties to determine when to send typing notifications.

In TextChanged event:
  If TypingStarted has not been sent
    Send TypingStarted
    Start a 2 second timer
      When the timer elapses, send TypingPaused unless TypingStopped was sent in the interim

Note the absence of the check on the text length…whoops, and also the absence of the TypingStopped notification. This has a strange effect in clients that support all three notification types: Pidgin turns a conversation tab orange when it gets TypingPaused and won’t reset it until TypingStopped (or a new message) is received, and chatting with a shaim client results in a rather orange tab a lot of the time. To compensate, a second timer is used:

In TextChanged event:
 ...
    When the timer elapses, if TypingStopped was not sent in the interim
      Send TypingPaused
      Start a 10 second timer that sends TypingStopped when it elapses

Conclusion: Don’t touch WPF input controls in tight loops, and AIM’s indelible mark

Is it a perfect solution? Well no, not completely, but the three-state notification model isn’t a perfect solution either. Most protocol plugins actually boil it down to only the TypingStarted and TypingStopped states, translating TypingPaused to TypingStopped. Solving this problem made me go back and think about why we’re even supporting the three-state model, and it turns out that shaim’s UI supports it because AIM supports it (shaim was originally AIM-only). It’s entirely possible that we’ll remove it from the framework at some point.

Written by Chris

March 22nd, 2008 at 4:51 pm

dotnetornot – the conversation that became a tool

without comments

Several weeks ago, an IT friend of mine was telling me about a horrific tool that some vendors had given him. This tool had two distinctive features: it had dozens of DLLs packaged with it, and it was slow as hell. My friend wanted to ngen the DLLs in order to squeeze some performance out of the tool, but in a hilarious twist, it seemed that not all of the DLLs were .NET asemblies, and those that were .NET were built against different versions of the Framework. He asked me if there was any way to tell what version of .NET an assembly used; I replied “Using Reflection? Yeah, probably” and sent him the following C# code:

namespace dotnetornot
{
  class Program
  {
    static void Main(string[] args)
    {
      System.Reflection.Assembly assm = null;
      try
      {
        assm = System.Reflection.Assembly.LoadFrom(args[0]);
        System.Console.WriteLine("This assembly uses .NET version {0}",
          assm.ImageRuntimeVersion);
      }
      catch (System.BadImageFormatException bifex)
      {
        System.Console.WriteLine("Not a .NET assembly");
      }
    }
  }
}

Copy-paste-compile, call it like “dotnetornot <path-to-file>”

Anyway, a couple nights ago, my friend was telling me about how he was dealing with this tool, and said “btw dotnetornot is amazing.” I had forgotten about it completely, but it seems he’s been using it to automate the process of calling the appropriate ngen on assemblies that support it. Nice :)

Written by Chris

March 19th, 2008 at 10:04 am

Posted in Development,General

shaim has moved

without comments

Not like anyone who actually cares doesn’t already know but for Posterity: shaim has packed up and left Bounty Source, our project host for many moons, and moved to http://shaim.net, hosting and page design generously provided by our very own project manager Greg. You can read about why we left on the news page.

Written by Chris

February 8th, 2008 at 4:51 pm

Posted in Development,General

Finding the phantom ArgumentException

without comments

Last night I knocked out a bug in shaim that had been annoying me for months. It had to do with the OscarLib driver, and changing the order of your contact list from another client. I’d finally gotten it fixed to the point where moving a contact from one group to another would work sometimes, seemingly depending on the order of the groups that were involved in the move, but the inconsistency was driving me insane.

A little background: Groups and contacts are represented in OscarLib by fairly simple objects:

class Group
{
  public string Name;
  public ushort ID;
  public ushort[] childIDs;
}

class Contact
{
  public string Name;
  public ushort ID;
  public ushort parentID;
}

When the AIM protocol sends out notice of a remote list update, it does so in several steps:

  1. Alert the client that the list is about to change
  2. Send a “contact removed from original group” notification
  3. Send a “contact added to new group” notification
  4. Tell the client that the original group has had its children changed
  5. Tell the client that the new group has had its children changed
  6. Send an “All done” message

Whew, that’s a lot of messages! The OscarLib driver needs to remember the order of events so that, when the “new group children changed” message comes in, it can add the contact to the new group in the appropriate order (people complain a lot when their list gets rearranged).

To effect this memory, the OscarLib driver maintains a list of groups and the contacts in those groups that have been updated.

Dictionary<Group, List<Contact>> outstandingGroupUpdates;

It keeps a list of contacts to update because it’s possible for the server to send multiple list updates before it sends the final group update, so they’re all performed at the same time when the server says it’s done. Let’s look at the message order again, and see what the driver does.

  1. Alert the client that the list is about to change
  2. Send a “contact removed from original group” notification. Remove item from shaim’s list
  3. Send a “contact added to new group” notification. Add the contact to outstandingGroupUpdates
  4. Tell the client that the original group has had its children changed
  5. Tell the client that the new group has had its children changed. If the group has a list of contacts to update in outstandingGroupUpdates, perform the list updates
  6. Send an “All done” message

That’s all the necessary background. The problem was happening in the second-to-last step, where the driver attempted to look in the Dictionary and see if there were any updates to perform:

void GroupChildrenChanged(Group group)
{
  if(outstandingGroupRequests.ContainsKey(group))
  {
    foreach(Contact contact in outstandingGroupRequests[group])
    {
      ProcessUpdate(group, contact);
    }
  }
}

The only clue I had was that when this code executed, and the driver was inclined to fail (random), the console would print

A first-chance exception of System.ArgumentException was thrown in mscorlib

…and the method would just stop. The exception didn’t ever make it up to user code, it just lived and apparently died in the Framework itself. But where the hell was it coming from? Putting in a lot of breakpoints and console prints, I found out that it was the Dictionary.ContainsKey method causing the exception. Well crap, it’s not supposed to throw ArgumentException!

The usual suspects – a null group being passed to the GroupChildrenChanged handler, the dictionary being in an invalid state, the group not successfully being added to the dictionary in the first place – all turned up bupkiss. I broke out Reflector and peered into mscorlib’s implementation of Dictionary to see if it was throwing an ArgumentException at any point and just handling it itself (though badly) – nothing. In fact, the only thing that could throw was the IEqualityComparer used by the Dictionary class to compare two objects to determine whether an object was, in fact, contained by the dictionary.

The IEqualityComparer was a bit of an enigma and a good candidate for Problem Child. The Framework contains four or five (I forget) implementations of an equality comparer that were chosen by the type of the generic objects it was comparing. It could be strings, bytes, comparable objects, or miscellaneous, and no way to tell which was being chosen at runtime*. The objects getting compared were my plain old Group objects, so I guessed that the miscellaneous comparer was being used, and Reflected into that. Surprise, no explicitly thrown ArgumentExceptions there either, but I wanted to rule out the miscellaneous comparer as the culprit entirely. I went through my Group and Contact objects and implemented IComparable on them, so the IEqualityComparer would choose its “comparable objects” implementation…and it started working, no other modifications necessary.

The moral of the story would seem to be to make sure that the keys for a Dictionary are IComparable, but that’s pretty weird. Every object has a default implementation of Equals, and since I didn’t need comparison above and beyond reference equality, that should have been sufficient. And I’m still not entirely sure where the ArgumentException was coming from…was it the default implementation of object.Equals in use by the Group object? Was it the generic IEqualityComparer being used by the Dictionary? Why was the exception not bubbling up to the user code? Madness!

* While I was investigating this, I read an MSDN blog post that indicated that the next version of Visual Studio would, in fact, be able to debug down into the Framework source. I could have stepped through live code until I found the problem. Oh well.

Written by Chris

October 4th, 2007 at 3:48 pm

Posted in Development,General

A trip down memory lane

with one comment

Last night I committed the 1000th revision to shaim, just one month shy of its two-year anniversary. What began as a cobbled-together .NET 1.1 application back in 2005 is now a .NET 3.0 plugin-based architecture comprising 13 projects and 125,000 lines of code. Looking back over the Subversion repository logs, I can remember landmarks both comic and tragic that got us where we are today.

  • Rev. 1, 698 days ago: Greg convinces me to use a Subversion repository hosted on his computer and #aim is imported to source control for the first time.
  • Rev. 13, 655 days ago: Code begins to appear to connect an AIM account, starting what will eventually become OscarLib.
  • Rev. 62, 598 days ago, three months after the project starts: OSCARlib, as it was known, is integrated into #aim. Plugins? Pfft, whatever.
First message window
  • Rev. 115, 579 days ago: An icon was made for #aim by Jason Crane.
Old icon
  • Rev. 146, 540 days ago: A critical problem in OscarLib was figured out while I was in the shower, beginning a long-standing tradition of figuring out problems with shaim in the shower and while asleep.
  • Rev. 160, 504 days ago: The old # icon was replaced with the current version.
Current icon
  • Rev. 174, 483 days ago: The original shaim-dev team convenes for the first time. We have a design meeting that results in the architecture that shaim is based upon to this day. Windows Presentation Foundation, which was at that time in beta, is chosen for as basis for the UI. “#aim” is renamed to “shaim,” and the project is moved onto the Bounty Source servers.
  • Rev. 276, 435 days ago: shaim 0.1 beta is released on Something Awful. It contains OscarLib and an early MSN plugin based on DotMSN.
  • Rev. 419, 385 days ago: shaim 0.2 beta is released. DotMSN support was dropped to focus on OscarLib development. Sometime between 0.1 and 0.2, a Yahoo! support plugin rose and fell. I had no recollection of this until today…
  • Rev. 451, 357 days ago: The first commit by a member of the Western Washington University senior design group, a team of four computer science students who used shaim development as their final project.
  • Rev. 653, 296 days ago: shaim 0.3 beta is released, with OscarLib supporting file transfers and other keen things.
  • Rev. 767, 179 days ago: DotMSN is dropped permanently, and MsnLib begins development as shaim’s MSN support plugin.
  • Rev. 801, 170 days ago: shaim 0.3.5 beta is released.

Then something something something, and then Rev. 1000, 1 day ago: Fixed up the “About…” window and updated the current shaim-dev team. Four current developers, ten developers over the course of the project, three people contributing art, countless dedicated and fearless testers, and that’s how shaim came to be. Here’s to another thousand revisions!

Written by Chris

April 18th, 2007 at 9:45 am

Posted in Development,General

StopLockRoll – a SuperPissed® Production

with 2 comments

In yet another fit of getting annoyed at something and doing something about it, I have created StopLockRoll – a small utility that sits in the notification area and pauses iTunes when the workstation locks. Brilliant!

StopLockRoll screenshot

StopLockRoll (107 KB)
Source – C# 2.0 (108 KB)

Written by Chris

March 20th, 2007 at 5:55 pm

Posted in Development,General