csammisrun

A rare situation

Archive for the ‘shaim’ Category

Blast from the past

with 2 comments

Extremely nerdy post ahoy!

The new version of the .NET Framework will be coming out relatively soon, and I spied this little gem on the BCL Team Blog: ObservableCollection<T> is moving into System.dll so implementers can avoid a dependency on WPF.

That would have been nice about a year and a half ago :(

Written by Chris

October 22nd, 2009 at 4:02 pm

Posted in Development,shaim

I’m all up outta here, part 3

with 2 comments

The even-stunninger conclusion to I’m All Up Outta Here parts one and two!

I’m just going to come out and say it: I’m burned out on shaim. As of right about now, I’ve been developing some incarnation of shaim for four years. I really enjoyed the experience that it netted me, as well as the satisfaction of helping loyal users as they reported bugs and endeavored to help me make it better, but now enough is enough. The site will remain up as long as Greg sees fit to leave it up; if it ever goes down, I’ll transition the codebase to another host so that anyone who wants to use any part of it may do so.

Thanks go out to everyone who’s contributed code, resources, or general support over the years. In no particular order:

  • Greg
  • CheeToS
  • saiyr
  • Joy
  • Andrew
  • The group from Western Washington University who contributed to shaim for their senior project
  • The dozens of people who’ve contributed bug reports and kudos

PEACE OUT

Written by Chris

April 16th, 2009 at 8:35 am

Lock ‘n’ Roll

with one comment

A computer is a box where decisions are made.

Phillippe, age 5

One of the computer decisions I make on a frequent basis is to get up and walk away from it, locking the workstation as I go. Sadly, Pidgin, the IM client I use at work, remains blissfully ignorant of my choice. Other IM clients automatically set the account status to “Away” when the workstation is locked, but Pidgin does no such thing…until now.

I’ve created a Pidgin plugin called Lock ‘n’ Roll that will set any active accounts to Away (with a user-defined message if applicable) when the workstation locks, and back to Available when it is unlocked. If an account has an existing away message set, Lock ‘n’ Roll leaves it alone. This is my first shot at Pidgin development so be sure to report any errors to so that I can fix them and make everyone’s life a little happier.

Download locknroll-1.0.zip, source and binary included. To install, copy “locknroll.dll” to the Pidgin plugins directory. To activate it, restart Pidgin and go to the Tools menu, select Plugins, and tick the checkbox next to “Lock ‘n’ Roll”

Written by Chris

January 22nd, 2009 at 9:18 am

You’ve got to be kidding me

without comments

Today I set up a Google Alert to tell me when someone talks about shaim. Not two hours later I got the first hit: “There is a chiyuv to speak out against r’shaim.” What?

The Hebrew r’shaim translates to wicked men.

Written by Chris

September 25th, 2008 at 1:33 pm

Posted in General,shaim

What thread do shaim events use? – or, Why status updates don’t make the UI cry

without comments

Fourth in the series of “how or why shaim does what it does”:

  1. WPF versus IM typing notifications
  2. WPF windows, in my Winforms application?
  3. .NET 3.0 functionality in .NET 2.0: a sorting, filtering, bindable list (Part Two)

Right on the heels of the two-parter on the BindingListCollection, this little spiel will continue the trend of describing the core contact list and its data structures, and how it all works with the UI. We’ll also touch on the threading model – or complete lack thereof – of the shaim core and its plugins.

A little GUI development background, but not much

A graphical UI (any graphical UI, not just shaim’s) can only be updated by the thread that created it. If another thread tries to update some text, or move a window, or change something’s color, the UI could deadlock and that is not good. The process of executing code on the UI thread is called invoking : any thread can call the Invoke method on a UI object, give it some code to execute, and that code will be executed on the UI thread so it can safely update the UI.

shaim’s hedonistic threading free-for-all

With one notable exception – which is the subject of this post – the shaim core doesn’t make much of an attempt to control the threading of its plugins. Each plugin is free to do whatever it wants, and shaim events aren’t guaranteed to be synchronized to anything in particular (the core does synchronize access to its data structures so that operations like ContactList.UpdateContactStatus will execute in the order they were called).

Let’s take OscarLib as an example. The initial login sequence of OSCAR is a pretty rigid call-response setup, so when the network socket receives data, it processes and sends a response on the same thread. The synchronous processing ensures that there aren’t any responses out of order from the packets that initiated them. This all changes, however, once the OSCAR session is initialized and asynchronous messages like status updates start coming in. Processing something like an incoming message may take a noticeable amount of time, and if it’s being processed on the same thread that received it, the socket might end up missing a packet. To solve this problem, the socket uses the ThreadPool.QueueUserWorkItem method to dispatch an incoming packet. The packet is processed by a random thread in the thread pool, a system-allocated collection of threads that are meant to be spawned for processing information in the background (that is, not in the main process, which in this case is the thread that owns the socket).

“Processed by a random thread” – so these events are not happening on the UI thread. Huh. Problems ahoy!

The easy part: dispatching shaim events in the UI

Each shaim event defined by the core defines a delegate that can handle itself (tee hee).

public class ShaimMessageReceivedEvent : ShaimIncomingEvent
{
    public delegate void Handler(ShaimMessageReceivedEvent);
}

When a plugin registers to handle a specific event, it uses a method that matches the signature of the event’s Handler delegate. We can define a very very simple incoming message handler registered by the UI plugin that will display the message to the shaim user:

private void HandleIncomingMessage(ShaimMessageReceivedEvent evnt)
{
    Conversation conv = FindConversation(evnt.Contact); // Gets a reference to the conversation window for this contact
    conv.AppendMessage(evnt.Message);  // Appends the text of the message to the conversation window
}

Simple, but it won’t work right. When this event comes in on a thread other than the UI thread – and it most assuredly will – the UI will lock up and WPF may even throw an exception and die unexpectedly and leave a ton of debt for its family to take care of, but it had no life insurance and the whole debacle ends in bankruptcy.

Let’s avoid that by dispatching the incoming event onto the UI thread:

// A simple delegate that will be initialized in a single method's scope
private delegate void AnonymousInvoker();

private void HandleIncomingMessage(ShaimMessageReceivedEvent evnt)
{
    AnonymousInvoker invoker = delegate
    {
        Conversation conv = FindConversation(evnt.Contact); // Gets a reference to the conversation window for this contact
        conv.AppendMessage(evnt.Message);  // Appends the text of the message to the conversation window
    };
    // Invoke the operation on the UI thread
    Application.Current.Dispatcher.BeginInvoke(invoker);
}

It’s a bit more verbose, and it has to be done for every incoming event, but it’s 100% thread-safe. It also has the benefit of removing the responsibility of marshalling every event onto the UI thread from the core. This is a desirable trait because it encapsulates UI responsibility within the UI plugin. The shaim framework might not even have an active UI plugin, so there’d be no reason for the core to have to perform the extra logic.

The hard part: dispatching binding events in the core

The above works well when the developer is defining their own event handlers, but it doesn’t solve the thread dispatch problem when something happens for which there isn’t an explicit shaim event. When there’s a direct binding between a data structure and the UI, and another thread updates the data structure, the binding attempts to update on the same thread that invoked the change and the UI craps itself. This presents a huge problem for the contact list and its constituent objects and collections, all of which are directly bound by the UI and updated by random threads spewing out of the plugins.

Well hell, there goes our fever-dream of the core not knowing anything about dispatching onto the UI. Fortunately, we can at least design the solution so that the core doesn’t have to know anything specific about the UI. It has to know two things: one, that the UI requires dispatching, and two, how to dispatch an event onto the UI thread. In turn, the UI has to expose information about whether it requires dispatching, and provide a generic mechanism for doing dispatch.

An interface is defined that provides a method to check whether if dispatch is required, and a method to do the dispatch:

/// <summary>
/// Specifies an interface for an object that can dispatch method calls onto the UI thread
/// </summary>
public interface IUIDispatcher
{
    /// <summary>
    /// Dispatches a delegated method onto the UI thread
    /// </summary>
    /// <param name="methodDelegate">The delegate to invoke on the UI thread</param>
    /// <param name="parameters">A list of parameters to pass to the method</param>
    void Dispatch(Delegate methodDelegate, params object[] parameters);
    /// <summary>
    /// Checks to see if the currently executed thread has access to the UI thread
    /// </summary>
    /// <returns><c>true</c> if a method can be executed in the context of the UI thread</returns>
    bool CheckAccess();
}

…and the UI plugin interface is extended to request one of these buggers from the UI plugin:

public interface IUIPlugin
{
    ...
    /// <summary>
    /// Gets a <see cref="IUIDispatcher"/> object to delegate method calls to the UI thread
    /// </summary>
    /// <returns>A new <see cref="IUIDispatcher"/></returns>
    IUIDispatcher CreateDispatcher();
}

Each UI plugin will define their IUIDispatcher differently, but the WPF plugin does it like so:

public class UIDispatcher : IUIDispatcher
{
    private DispatcherPriority dispatchPriority;

    /// <summary>
    /// Initializes a new UIDispatcher
    /// </summary>
    /// <param name="dispatcherPriority">The priority at which to dispatch method calls</param>
    public UIDispatcher(DispatcherPriority dispatcherPriority)
    {
        this.dispatchPriority = dispatcherPriority;
    }

    #region IUIDispatcher Members

    public void Dispatch(Delegate methodDelegate, params object[] parameters)
    {
        if (Application.Current != null)
        {
            if (parameters.Length > 0)
            {
                Application.Current.Dispatcher.Invoke(dispatchPriority, methodDelegate, parameters[0]);
            }
            else
            {
                Application.Current.Dispatcher.Invoke(dispatchPriority, methodDelegate);
            }
        }
    }

    public bool CheckAccess()
    {
        return Application.Current.Dispatcher.Thread == System.Threading.Thread.CurrentThread;
    }

    #endregion
}

Using the IUIDispatcher in the BindingListCollection

Let’s take a look back at the BindingListCollection. It raises binding events in response to things like adding or removing items, as well as moving items around in response to status or name updates. Since those updates may be coming from some crazy random thread, it’ll need to use the IUIDispatcher to raise those events. The following is pseudocodeish, but it illustrates the point; consult the actual BindingListCollection code for details on the move operation and how the IUIDispatcher gets assigned.

private IUIDispatcher uiDispatcher;
...
private delegate void MoveInListHandler(T item);

void MoveInList(T item)
{
    // If there is no UI dispatcher or we're on the right thread...
    if(uiDispatcher == null || uiDispatcher.CheckAccess())
    {
        // Do the MoveInList action, finding new indexes and moving the item
    }
    else
    {
        // There is a UI dispatcher to contend with, and we're on the wrong thread
        // Invoke the current method on the UI thread, then the above code will execute correctly
        uiDispatcher.Dispatch(new MoveInListHandler(MoveInList), item);
    }
}

Presto! This method is used throughout the BindingListCollection and the ContactList to ensure directly bound data members are updated on the correct thread, and no UI plugin has to be concerned about where updates are coming from.

Ensuring events are invoked correctly is a pain in the ass, but a wholly necessary one. The IUIDispatcher is the best method I could design, but if any readers have ideas about how shaim could institute a threading model that doesn’t involve *every* plugin operation being executed on the UI thread, I’d love to hear about it in the comments.

Written by Chris

July 25th, 2008 at 10:03 am

.NET 3.0 functionality in .NET 2.0: a sorting, filtering, bindable list – Part Two

with one comment

The thrilling conclusion to Part One!

When we left our hero, the BindingListCollection, we had a nicely serviceable collection that could be bound, sorted, filtered, and was .NET 2.0 compliant. New items added into the list were sorted and filtered correctly, and life was wonderful…but it was not perfect yet.

Ch-ch-ch-ch-changes (don’t want to be a static list)

If you have a contact list with more than zero people on it, chances are there are going to be events that alter the state of the list in some way. Contacts change status, occasionally their names change, the number of contacts in a group can change, and so on. A contact list is a highly mutable structure, and we want the data structure containing it to be able to respond dynamically.

I mentioned in part one that the BindingListCollection implements INotifyPropertyChanged, so that anything that binds to it can be made aware when a property on the collection changes. I also mentioned, in an off-hand way, that all good databound classes implement that interface. The core data structures that make up the shaim contact list – ContactGroup and MetaContact – implement INotifyPropertyChanged. Maybe the BindingListCollection should take advantage of this! We’re going to require that anything contained in a BindingListCollection implements INotifyPropertyChanged by putting a restriction on the generic type used by the container.

public class BindingListCollection : BindingList, INotifyPropertyChanged where T : class, INotifyPropertyChanged

Now that we know for a fact that all items being inserted into the list will have the PropertyChanged event, we can update InsertItem to attach event handlers when a new item is added:

...
    // Insert the item into the list
    if (index != -1)
    {
        if (index > base.Count)
        {
            index = base.Count;
        }
        base.InsertItem(index, item);
    }

    INotifyPropertyChanged propitem = item as INotifyPropertyChanged;
    if (propitem != null)
    {
        propitem.PropertyChanged -= item_PropertyChanged; // Don't attach it twice
        propitem.PropertyChanged += item_PropertyChanged;
    }

    // Fire the AddingNew event
    OnAddingNew(new AddingNewEventArgs(item));
...

The event handler is removed in RemoveItem to avoid stray references that prevent garbage collection.

Now let’s think about the event handler method, item_PropertyChanged. The PropertyChanged event has only one parameter: the name of the property that changed on the item that raised the event. We don’t want to muck with the list on every property change, though – this is where the BindingListCollection starts to get specialized for shaim. With a minimal amount of effort, it could be made to work in generic applications.

shaim supports sorting/filtering by a metacontact’s display name (alphabetic sort), a metacontact’s status (status sort, or hide offline contacts), a group’s display name (alphabetic sort), or a group’s count (hide empty groups). With that in mind, we can listen for only those specific properties, and fiddle with the list contents only if one of those changes.

void item_PropertyChanged(object sender, PropertyChangedEventArgs e)
{
    T item = sender as T;

    // If the list isn't sorted or filtered, no rearrangement is necessary
    if (!isSorted && !isFiltered)
    {
        return;
    }

    // Lists of groups can be sorted/filtered by display name and online count,
    // lists of metacontacts by display name and status
    if (e.PropertyName == "DisplayName" || e.PropertyName == "Status" || e.PropertyName.EndsWith("Count"))
    {
        MoveWithinList(item);
    }
}

MoveWithinList: thanks for being terrible, WPF TreeView

Before I start raving incoherently about a bug that took for-freakin’-ever to track down and squash, I shall present and explain the code listing for MoveWithinList and its support methods:

/// <summary>
/// A constant value returned by FindInCurrentList
/// when the requested item is not in the list
/// </summary>
private const int ITEM_NOT_IN_LIST = -1;

/// <summary>
/// Move an item within the list by its current properties
/// </summary>
private void MoveWithinList(T item)
{
    bool fireItemsChangedEvent = false;
    int originalPosition, newPosition;

    // Remove the item at its current position
    originalPosition = FindInCurrentList(item);
    if (originalPosition != ITEM_NOT_IN_LIST)
    {
        Items.RemoveAt(originalPosition);
        fireItemsChangedEvent = true;
    }

    // Reinsert it at its new position, maybe
    newPosition = PlaceByCurrentComparison(item);
    if (newPosition != ITEM_NOT_IN_LIST)
    {
        Items.Insert(newPosition, item);
        fireItemsChangedEvent = true;
    }

    if (fireItemsChangedEvent)
    {
        // Figure out which list changed event to fire, given the change
        // in the active list.

        if (IsValidPosition(originalPosition) && IsValidPosition(newPosition)
            && originalPosition != newPosition)
        {
            // An item was valid, it's still valid but it moved, fire a...something
            // [trombone making that wah-wah-waaaaaah sound]
        }
        else if (IsValidPosition(originalPosition) && !IsValidPosition(newPosition))
        {
            // An item was valid, now it's not, fire a Delete event
            OnListChanged(new ListChangedEventArgs(ListChangedType.ItemDeleted, originalPosition));
        }
        else if (!IsValidPosition(originalPosition) && IsValidPosition(newPosition))
        {
            // An item was invalid, now it is, fire an Add event
            OnListChanged(new ListChangedEventArgs(ListChangedType.ItemAdded, newPosition));
        }

        if (PropertyChanged != null)
        {
            PropertyChanged(this, new PropertyChangedEventArgs("Items"));
        }
    }
}

/// <summary>
/// Returns a value indicating whether the given position is valid for the list
/// </summary>
private static bool IsValidPosition(int position)
{
    return position > ITEM_NOT_IN_LIST;
}

/// <summary>
/// Returns the index of an item in the currently displayed list
/// </summary>
private int FindInCurrentList(T item)
{
    return Items.IndexOf(item);
}

The logic here is pretty simple. When an item’s property changes, one of four things will happen: it may be moved by the current sort, it may become visible if it was previously filtered, it may become hidden by the current filter, or it might do nothing. MoveWithinList locates where the item is currently, where it’s going (if anywhere), does the rearranging, and fires a list changed event to tell binders what just happened. This completes the mimicry of INotifyCollectionChanged.

Almost completes it, that is. Note the comment in the case where the item moves in the currently visible list. This is not as intuitive as one might hope. If you look up the MSDN documentation for the ListChangedType enumeration, you’ll see ItemAdded and ItemDeleted – both of which are already used by MoveWithinList – and also ItemMoved. How simple! In the mystery event area, just put:

OnListChanged(new ListChangedEventArgs(ListChangedType.Move, newPosition, originalPosition));

…and just wait for the horrific exceptions to roll in!

Unexpected collection change action 'Move'.
  at System.Windows.Controls.TreeViewItem.OnItemsChanged(NotifyCollectionChangedEventArgs e)
  at System.Windows.Controls.ItemsControl.OnItemCollectionChanged(
     Object sender, NotifyCollectionChangedEventArgs e)

Oh boy! The WPF TreeView apparently doesn’t support moving items under a single TreeViewItem. I wonder why this is? Let’s visit our good friend, Reflector:

protected override void OnItemsChanged(NotifyCollectionChangedEventArgs e)
{
    switch (e.Action)
    {
        case NotifyCollectionChangedAction.Add:
            return;

        case NotifyCollectionChangedAction.Remove:
        case NotifyCollectionChangedAction.Reset:
            if (this.ContainsSelection)
            {
                TreeView parentTreeView = this.ParentTreeView;
                if ((parentTreeView == null) || parentTreeView.IsSelectedContainerHookedUp)
                {
                    return;
                }
                this.ContainsSelection = false;
                this.Select(true);
            }
            return;
    }
    throw new NotSupportedException(SR.Get(SRID.UnexpectedCollectionChangeAction, new object[] { e.Action }));
}

Going back up the call stack of the exception, I see that ListChangedType.ItemAdded gets changed to NotifyCollectionChangedAction.Add, ItemDeleted gets changed to Remove, and Reset stays the same. ItemMoved, which gets changed to NotifyCollectionChangedAction.Move, is simply not supported by WPF’s TreeView. C’est la vie, I suppose, let’s just go with what is supported. Instead of the ItemMoved ListChanged event, do a reset instead:

if (IsValidPosition(originalPosition) && IsValidPosition(newPosition)
    && originalPosition != newPosition)
{
    OnListChanged(new ListChangedEventArgs(ListChangedType.Reset, -1)); // <-- f-f-f-frickin hacks!
}

Fortunately this happens less than you might think. Most of the time, a change event happens slow enough that it triggers an add and delete instead of a full reset.

Getting at the real stuff

Consumers of the BindingListCollection may still want to know about what's underneath the filtering and sorting, so we'll expose a couple properties to let them get at that data (but in a read-only sense).

/// <summary>
/// Return the total number of items in the list, visible or not
/// </summary>
public int TotalCount
{
    get
    {
        if (isSorted || isFiltered)
        {
            return unsortedList.Count;
        }
        else
        {
            return Items.Count;
        }
    }
}

/// <summary>
/// Gets a read-only view of the underlying list without filtration or sorting
/// </summary>
public ReadOnlyCollection<T> UnderlyingList
{
    get
    {
        if (isFiltered || isSorted)
        {
            return new ReadOnlyCollection<T>(unsortedList);
        }
        return new ReadOnlyCollection<T>(Items);

    }
}

To finish up, we still have to advertise that the BindingListCollection can do these fun things, so binders can take advantage of it. This is as simple as overriding the various "Supports____" properties of the BindingList and returning true.

That's it! For reals this time!

Oh wait the basil-lime sorbet recipe

Yeah it sucked. Too much simple syrup, not enough lime juice. The basil was good though.

THE END.

Written by Chris

July 17th, 2008 at 9:53 am

.NET 3.0 functionality in .NET 2.0: a sorting, filtering, bindable list

with 3 comments

The third in a series of write-ups on some of the odder parts of shaim.

  1. WPF versus IM typing notifications
  2. WPF windows, in my Winforms application?

As I described in the last write-up, shaim is organized around a central core. The core provides services such as preference loading/saving, loading plugins and dispatching inter-plugin events, and a factory for proxied sockets. It also holds the most central data structure to any instant messaging application: the contact list. This post is going to talk about creating a data structure that replicates the functionality of a .NET 3.0 INotifyCollectionChanged collection in .NET 2.0-compatible code. Why would you want to do that in the first place? All shall be made clear…

Representing and displaying the contact list

For most IM systems and for the purposes of this discussion, a contact list is hierarchical. Stemming from the root of the list are groups, each of which can contain one or more contacts

Contact List
|
+-- Group
|   |
|   +-- Contact
|   +-- Contact
|
+-- Group
    |
    +-- Contact

Hierarchical data of this nature is naturally represented using a tree view, and shaim’s UI plugin is no exception. It uses the WPF TreeView control, which is capable of binding to a list-of-lists and displaying the data in a pleasing fashion. If you bind to a collection that implements the INotifyCollectionChanged interface, the TreeView will even do such kind things as update the view automatically when the bound list changes.

A challenge!

One of the driving goals of the project is to maintain a core that can be compiled on Mono, which means the core uses absolutely no .NET 3.0 components or data structures. The modularity of the plugin design still lets the framework utilize plugins that are based on .NET 3.0 technology, but as long as the core maintains its purity, it’s all good in our hood. Tra la la, happy days…oh shit, INotifyCollectionChanged is a .NET 3.0 interface. The contact list can’t use it, the TreeView won’t update itself properly, chaos ensues, and I’m lying in a ditch somewhere having been beaten to death by hired goons.

Well that’s alright, that’s okay, no one ever said life was simple. We’ll just implement a collection with similar – and compatible – behavior using .NET 2.0. In fact, since people like to do silly things like organize their data alphabetically (pfffft), let’s add sorting logic to it. And while we’re being all nutty, let’s add a filter so users can remove items that meet certain criteria, then people can do such as not display empty groups or offline contacts. Above all, it has to be painless from the UI’s point of view: we don’t want any of this logic floating up into the UI, where it would have to be replicated by any new UI plugin.

So this collection has to be:

  • .NET 2.0 compatible
  • Bindable by WPF and Winforms
  • Sortable
  • Filterable
  • Hell, let’s make it generic while we’re at it

We shall call it: BindingListCollection!

The basics of the BindingListCollection

First things first, we have to find .NET 2.0 structures and interfaces that we can start with that’ll let us fake the funk appropriately.

Like all good classes that are bound in some way, the BindingListCollection will implement INotifyPropertyChanged. This interface is similar to the verboten INotifyCollectionChanged, in that it notifiers consumers of a change in the implementing object, but instead of a change on the contents of a collection the INotifyPropertyChanged interface notifies of changes in a single property. Perhaps we can fool the TreeView into refreshing its view of the collection by notifying of a property change on the Items property! In fact we can, this does work, but it’s pretty horrific. Anything binding to a specific property completely refreshes when a property change happens, which gets expensive and slow when you’re talking about a list of any sort of appreciable size. The INotifyCollectionChanged interface gets around this by raising an event that specifies exactly what action was taken (adding or removing an item, for instance) and where in the list it happened, so the entire thing isn’t munged by what should be a quick change.

Is there an interface or a class in .NET 2.0 that is clever like the INotifyCollectionChanged interface? Turns out that there is: BindingList. It fits all of our requirements: it’s .NET 2.0, supports binding, has support for sorting and filtering, and is generic. So begins our new type, BindingListCollection.

public class BindingListCollection<T> : BindingList<T>, INotifyPropertyChanged

Now whenever we manipulate the collection (by adding, removing, filtering, or sorting items), raising the ListChanged event will cause whatever’s binding to it to update correctly and specifically, just like INotifyCollectionChanged.

Filtering and sorting: adding support for all this ballyhoo

The BindingList provides support for filtering and sorting, but leaves it to you to implement the routines to do it. Let’s make this super-extensible, and create methods to filter and sort using generic delegates for the purpose. Consumers of the BindingListCollection can create custom methods to sort and filter, then pass it in and let the BindingListCollection do the rest.

private bool isSorted;
private bool isFiltered;
/// <summary>
/// The sorting comparison currently in use
/// </summary>
private Comparison currentComparison;
/// <summary>
/// The filtering predicate currently in use
/// </summary>
private Predicate currentFilter;

/// <summary>
/// Sorts the list by the specified property in the specified direction
/// </summary>
public void Sort(Comparison sortCriteria)
{
    isSorted = true;
    currentComparison = sortCriteria;
    ApplyListAlterations();
}
/// <summary>
/// Filters the list contents by the specified predicate
/// </summary>
public void Filter(Predicate filterCriteria)
{
    isFiltered = true;
    currentFilter = filterCriteria;
    ApplyListAlterations();
}
/// <summary>
/// Applies the current filter and sort to the list
/// </summary>
private void ApplyListAlterations()
{
    List listref = Items as List;

    Items.Clear();
    listref.AddRange(unsortedList); // An original copy of all items - explained below

    if (isFiltered)
    {
        // Perform filtering on the current item list
        for (int i = 0; i < Count; i++)
        {
            if (!currentFilter(Items[i]))
            {
                Items.RemoveAt(i--);
            }
        }
    }

    if (isSorted)
    {
        // Sort the current item list
        listref.Sort(currentComparison);
    }

    // Tell all bindings to reset completely
    OnListChanged(new ListChangedEventArgs(ListChangedType.Reset, -1));
}

That was it! We now have a filterable sortable bindable generic collection...but it sucks! If you add a new item to the BindingListCollection, it doesn't get sorted or filtered. Right now a consumer has to call Sort and Filter again every time an item is added, which causes a complete reprocessing and binding reset, and that is e-x-p-e-n-s-i-v-e. We have to be clever to achieve awesomeness.

The essence of filtering is removing items from the list, but we have to save those items somehow; if we didn't, every time the filter was changed, or removed, or the item changed so that the filter no longer applies, the item would have to be re-added. The essence of sorting is changing the order in which items are displayed, but we want to preserve the original order. Both of these can be accomplished by maintaining a backing list - a list of all the items, exactly as they're added, that can be used to restore order when the sort or filter changes. We've already seen this in the BindingListCollection: the 'unsortedList' mentioned in ApplyListAlterations().

/// <summary>
/// The underlying list of manually inserted items
/// </summary>
private List<T> unsortedList = new List<T>();
/// <summary>
/// The index at which the item was originally inserted by the list controller
/// </summary>
private readonly Dictionary<T, int> insertIndicies;

The backing list must be maintained whenever items are added or removed from the BindingListCollection, so the InsertItem and RemoveItem methods are overriden to add to both the backing list and the main list. InsertItem is not terribly complicated:

/// <summary>
/// Inserts the specified item in the list at the specified index.
/// </summary>
protected override void InsertItem(int index, T item)
{
    // Maintain the backing list
    if (index > unsortedList.Count)
    {
        unsortedList.Add(item);
    }
    else
    {
        unsortedList.Insert(index, item);
    }

    // Add the requested index to the item-index map
    insertIndicies[item] = index;

    // Insert the item into the list
    if (index != -1)
    {
        if (index > base.Count)
        {
            index = base.Count;
        }
        base.InsertItem(index, item);
    }

    if (PropertyChanged != null)
    {
        PropertyChanged(this, new PropertyChangedEventArgs("Items"));
    }

    // Fire the AddingNew event
    OnAddingNew(new AddingNewEventArgs(item));
}

I'm not going to specifically list out RemoveItem here, but it's pretty simple: Remove the requested item from the backing list and insertIndicies, remove it from the base list, and that's that.

Filtering and sorting: what goes where?

Even though every item added to the BindingListCollection ends up in the backing list, it may not end up in the requested insert position if a sort is applied, and it may not end up in the main list at all if a filter is applied. The BindingListCollection has to compute an insertion index that makes sense according to the current sort and filter, and that's done using a new-fangled method, PlaceByCurrentComparison:

/// <summary>
/// Returns the index at which an item would be inserted by the current sort comparison
/// and current filter
/// </summary>
/// <returns>The index at which to insert the item, or -1 if it would not be in the currently filtered list</returns>
private int PlaceByCurrentComparison(T item)
{
    // If the item doesn't match the current filter, don't add it
    if (isFiltered && !currentFilter(item))
    {
        return -1;
    }

    int insertindex = 0;
    if (currentComparison != null)
    {
        // Get the index in the sorted list
        while (insertindex < Count && currentComparison(item, Items[insertindex]) >= 0)
        {
            insertindex++;
        }
    }
    else
    {
        // Check the originally requested insertion index, put it as close as possible
        if (insertIndicies.ContainsKey(item))
        {
            while (insertindex < Count &&
                (insertIndicies.ContainsKey(Items[insertindex]) && insertIndicies[item] >= insertIndicies[Items[insertindex]]))
            {
                insertindex++;
            }
        }
    }

    return insertindex;
}

...and the InsertItem method gets updated:

...
    // Add the requested index to the item-index map
    insertIndicies[item] = index;

    // If the list is sorted, find the correct insertion point for the sorted order
    if (isSorted || isFiltered)
    {
        index = PlaceByCurrentComparison(item);
    }

    // Insert the item into the list
    if (index != -1)
    {
        if (index > base.Count)
        {
            index = base.Count;
        }
        base.InsertItem(index, item);
    }
...

This seems pretty nice now: items are inserted into the list, their requested insert indexes are saved, but they may or may not be displayed sorted or at all, and the BindingListCollection takes care of all of it. The caller sees only this:

BindingListCollection<ContactGroup> groupList = ...;
groupList.Sort(someAlphabeticComparison);
groupList.Add(new ContactGroup("Buddies"));
groupList.Add(new ContactGroup("Amigos"));
groupList.Add(new ContactGroup("Wise-ass jerks"));

I am so done with typing right now

This is getting pretty ridiculously long for a blog post, so I'll finish off the discussion later this week. Still to come: Live updating of the filter and sort as items change, overriding properties to make sure the bound UI object knows what can be expected, and a failed recipe for basil-lime sorbet.

Written by Chris

July 16th, 2008 at 1:38 pm

WPF windows, in my Winforms application?

with one comment

In the last post about some of the odd things that shaim does, I talked about how WPF’s rich text input makes for laggy typing and how we lessened the impact. Today I’ll talk about how one should be grateful that any text input happens in shaim at all, due to the slightly strange way that WPF is used by the application.

Core ‘n’ plugins

The shaim architecture is plugin-based. There’s the core (shaim.exe, a Winforms Application), and several plugins that comprise the bulk of the application’s functionality. Many people don’t realize that the UI – the contact list, conversation windows, all of that noise – is a plugin itself, specifically a WPF Class Library.

When we were designing shaim back in the bad old days, we knew that we were going to use WPF for the UI, so why isn’t the core a WPF application? The answer is compatibility: WPF is only supported on Windows XP SP2 and up. We wanted to build shaim so that developers could make UI plugins with different windowing toolkits so it could run on other platforms. This has actually worked out in practice, as the core is fully .NET 2.0 compatible and there’s at least one community-driven UI plugin in the works (using GTK#).

Two event models

About the only thing that Winforms and WPF have in common is that they’re both Windows GUI toolkits. Winforms follows the general Win32 GUI event model: An Application has a message pump that raises events (mouse click, text typed, window moved, etc.), developers handle events as they are raised from the message pump, the world keeps on spinning. There are niceties to the process, but at its root that’s all there is to it. It’s not altogether difficult to see how Winforms is a shell over the Win32 API.

WPF has a different event model altogether, called routed events. Although the concept of direct events still exist, there are also now tunneling and bubbling events. The gist of it is this: The user types text into a text field, which generates a PreviewKeyDown event from the root element. This event “tunnels” down the visual tree until it gets to the text field, which then generates a KeyDown event, which “bubbles” back up the visual tree until it gets to the root element. At any point along the visual tree, either type of event can be handled.

Where all the keyboard events at?

When shaim’s UI and WPF itself were still very young – we’re talking like January / February 2006 – we noticed that we couldn’t type into any text input fields. Buttons clicked, windows moved, but no keyboard events were fired at all. Attaching event handlers to all the key press events in the visual tree, we found that all the tunneling Preview* events were being raised, but none of the corresponding bubbling events were happening, and no text would appear in the input field.

As much fun as having an IM client that you couldn’t use to send messages seemed, we decided to try hosting a Winforms text field control in one of the UI windows to see what would happen. Lo and behold, it worked just fine. Great, except at the time hosting was slow and rather bloated, not to mention it felt like a cheap hack. What the hell was wrong with WPF text input?

Two! Two Application objects! Ha ha ha!

After a lot of experimenting and talking to a Microsoft engineer (I forget who, sorry), we finally got it diagnosed. It was completely shaim’s fault for being architected as a Winforms application that launched WPF windows, instead of a straight WPF application or even a Winforms application that lauched Winforms windows that contained a Winforms->WPF host. There was no WPF message pump, so WPF-style routed events couldn’t happen correctly. Direct events, like clicking and moving windows, were still fine. The Winforms text field worked because the shaim core hosts a Winforms message pump in its Application object.

Okay, so give the UI plugin a message pump and it’ll be happy…but starting a message pump by calling Application.Run() is a blocking call. If the Winforms Application started a WPF Application, it couldn’t do much else. The stunning workaround: Start the WPF Application on its own sweet time.

void StartWpfApplication()
{
  application = new Application();
  application.Startup += application_Startup;

  AnonInvoker invoker = delegate
  {
    // Some slow operation that we didn't want to block core startup
    application.Run();
  };
  application.Dispatcher.BeginInvoke(DispatcherPriority.Normal,
    invoker);
}

void application_Startup(object sender, StartupEventArgs e)
{
  // signal the core that the UI's ready to roll
}

And that is why shaim users should be damn grateful that they can type in shaim at all

Written by Chris

April 4th, 2008 at 2:53 pm

JabberLib 0.1 released

without comments

I’m pleased to announce that JabberLib, a .NET library for connecting to XMPP (Jabber) servers, has just been released for the very first time on the Internet that you are reading! Right now!

Download the 0.1 release here

Who is this for?

JabberLib, similar to OscarLib and MsnLib, is a component for .NET developers to add Jabber messaging capability to their applications. It is written in C# and is .NET 2.0- (and Mono-) compatible.

Can I use this in my closed-source application that I sell for cash monies?

Sure, go right ahead! JabberLib is released under the MIT License and is freely available to all developers without restriction (or guarantee).

How can I add this to my project?

Quick and easy:

  • Download the 0.1 binary release and extract it to your project directory
  • Right-click on your project in Visual Studio and choose “Add reference…”; go to the “Browse” tab and select JabberLib.dll from wherever you extracted it.
  • See the example code to learn how to instantiate and manipulate a JabberLib Session.

I found a bug / I want to request a feature

Bug reports and feature requests can be logged on the JabberLib project site.

I want to contribute code, kind words, or death threats!

Stop on by #shaim on irc.freenode.net and chat with the developers. Most of us live in the East Coast / Central United States, but someone is always idling in the channel so feel free to leave a message.

Original posting on shaim.net

Written by Chris

March 28th, 2008 at 9:31 am

WPF versus IM typing notifications: how to type a 10 word sentence, get some coffee, and continue watching it render

with 5 comments

Hello and welcome to what I hope will be several write-ups on some of the stranger things that happen in shaim. As the project barrels towards its next significant release and we gain both developers and users, the system as a whole is undergoing something of a critical examination. Today’s topic concerns typing in the conversation window, something that until recently has been an extremely slow experience and every shaim user feedback to date has called out as being a barricade to using it on a day to day basis.

WPF and text input

shaim’s UI is written in Windows Presentation Foundation, which is pretty great for displaying all manner of things in very shiny ways, but has some problems with input and rich content input in particular. shaim, for better or for worse, uses a great deal of rich content input by way of the RichTextBox (RTB) class, extended by shaim to include support for HTML.

“What sort of problems does WPF have with text input?”

In general, it’s slow. In an effort to support everything under the sun, the WPF implementation of the RTB seems to have outgrown itself. Projects like SharpDevelop that require specialized text editing with rich content support tend to roll their own. There’s a noticeable lag between typing a stream of characters and seeing the results appear, and doing anything in an event handler on text input events naturally slows the process a bit more.

shaim and typing notifications

Most IM networks support the concept of typing notifications, small notification packets that indicate when a user is typing to another user. In general, this is a pretty simple concept: send a notification when typing starts, send one when typing stops. If the actual message is received, the client can assume that means typing has stopped (until the next notification is received). shaim’s typing notification support is based around AIM’s very special implementation of the feature, which features not two but three notification types:

  • Typing started: pretty clear
  • Typing paused: there is text in the user input field, but it hasn’t been sent yet
  • Typing stopped: the text field has been cleared

shaim handles the RTB’s TextChanged event to provide support for sending typing notifications. The basic flow is as follows:

In TextChanged event:
  Is text blank?
    Send TypingStopped
  If text is not blank and TypingStarted has not been sent
    Send TypingStarted
    Start a 2 second timer
      When the timer elapses, send TypingPaused unless TypingStopped was sent in the interim

Pretty simple, and it’s hard to boil that down any more with the three-state model. But it was dog slow, even slower than usual, and here’s why:

Is text blank?

It turns out that this is a bit more of a puzzler than you might expect. The RTB doesn’t expose a Text property, unlike a normal TextBox, but that fact itself isn’t unexpected: an RTB can hold markup, images, and all sorts of things that don’t belong in a string. Even so, it would be nice to have the plain text for various reasons. shaim’s RichEditControl fakes the funk by exposing a PlainText property that extracts the text content from the RTB.

So the test for blank text became

if (UserInput.PlainText.Length == 0)

Great, until you realize that this is being called in an event handler that is raised on every text change event – every single keystroke. The text extraction process isn’t cheap, and profiling showed that this caused a great deal of the text entry slowdown.

The next idea was to give the RichEditControl a new property, IsEmpty, that simply compared the RTB’s ContentStart and ContentEnd TextPointers to see if they were equal. If they were equal, the content was zero-length and the RTB was empty.

if (UserInput.IsEmpty)

Though it was indeed faster code-wise, there was still a very perceptible typing lag.

Okay, just forget it

It turns out that accessing any of the RTB properties from an event handler chain that is updating those properties isn’t cheap and was causing what amounted to unacceptable lag. The final solution? Don’t use the editor’s properties to determine when to send typing notifications.

In TextChanged event:
  If TypingStarted has not been sent
    Send TypingStarted
    Start a 2 second timer
      When the timer elapses, send TypingPaused unless TypingStopped was sent in the interim

Note the absence of the check on the text length…whoops, and also the absence of the TypingStopped notification. This has a strange effect in clients that support all three notification types: Pidgin turns a conversation tab orange when it gets TypingPaused and won’t reset it until TypingStopped (or a new message) is received, and chatting with a shaim client results in a rather orange tab a lot of the time. To compensate, a second timer is used:

In TextChanged event:
 ...
    When the timer elapses, if TypingStopped was not sent in the interim
      Send TypingPaused
      Start a 10 second timer that sends TypingStopped when it elapses

Conclusion: Don’t touch WPF input controls in tight loops, and AIM’s indelible mark

Is it a perfect solution? Well no, not completely, but the three-state notification model isn’t a perfect solution either. Most protocol plugins actually boil it down to only the TypingStarted and TypingStopped states, translating TypingPaused to TypingStopped. Solving this problem made me go back and think about why we’re even supporting the three-state model, and it turns out that shaim’s UI supports it because AIM supports it (shaim was originally AIM-only). It’s entirely possible that we’ll remove it from the framework at some point.

Written by Chris

March 22nd, 2008 at 4:51 pm