Here I will post problems I and my colleagues met and solutions we found.

Thursday, January 15, 2009

Smart Clients

About four years ago I heard about Smart Clients. Back then this term had particular meaning. Application can be considered Smart Client applications if

  • It's a desktop application with rich user interface.
  • As desktop application, it can work in disconnected mode.
  • When connected, additional functionality may be available.
I thought that it was a great idea and truly believed that such kinds of application would become highly popular. Instead, what we can see is an unprecedented rise of web applications.

I still like all these ideas, and still think that the idea of smart client application is valid. But it worth to analyze what went wrong with them, and why web became so popular. Well, analysis may be a too strong word, just some thought.

One of the reasons, I believe, is that it's much easier to develop web application than occasionally connected desktop application. There are so many people involved in web development now, that the choice of technologies, frameworks and examples is huge. In result, there is a new generation of developers who never developed anything by web applications. And these process feeds itself, the more popular web development is, the easier it becomes.

Another reason is that desktop applications were not that good looking. Somehow web became more visually appealing. Why did it happen? For one thing, desktop application development stuck with Win32 windows. (I have to mention that my view are based on my experience, which is development for Windows OS). As much as I love .NET, all we had for user interface were WinForms, and this is just the wrapper around old Win32 windows, where presentation cannot be easily separated from development, by which I mean coding. It's different in Web. HTML, with it's styles, allowed to developers to develop, and designers to design. And these are different talents.

Finally, it appears that many applications just don't need neither rich UI nor ability to work being disconnected. There is just no benefits for them in that.

So, where do we go now? Is it time to bury the idea? After all, even the term Smart Client is deluded. Every desktop application can be named Smart Client today, and I even heard the term applied to a web application with Java applet as a client, that just cannot work disconnected at all. I would argue that we should not just write it off yet. Here is why.

Trends. Let's look at where web applications are going. Many of them remind desktop applications now. Not only very rich UI is available through JavaScript, Flash, or Silverlihgt, but because they are starting to store their data on a client machine. Just look at such things as Google Gears, or Isolated Storage in Silverlight, I am sure there are others.

WPF gives desktop developers what web developers had long time ago. Presentation is finally separated from implementation. Layout of application is not part of the code anymore. With tools like Expression Blend designer can do his job while developer does his. And the results can finally be better than what they are in Web. There are concerns here. After all, WPF is not exactly new already, and adaptation is not going really fast. I would suggest that one of the reason is that XAML is too complicated, another is there are no tools on the market that could much those available for web designers. Expression Blend is definitely not there yet. Hopefully, the latter is just the matter of time, and once those tools are available, the complexity of the XAML will be hidden from designers.

Bottom line, I am optimistic about Smart Client applications. The deployment can be simplified with technologies like ClickOnes, or similar. Automatic updates are already a must. Disconnected mode is not that critical now, but having local cache is easier to do with existing light weight databases. WPF will bring pretty look. Sure, there are so many applications that just belong to web, there are those that can be better with smart clients, particulary business applications (emails, data entry applications, any office application).

Monday, January 12, 2009

Lazy loading for CSLA lists to support Server mode in DevExpress Grid. Part 4

The last parts are processing DataFetch based on LazyLoadingCriteriaBase class, which is defined as:



/// <summary>
///
/// </summary>
public enum LoadingCommand
{
/// <summary>
///
/// </summary>
DoNothing,
/// <summary>
///
/// </summary>
FetchData,
/// <summary>
///
/// </summary>
LoadIds,
/// <summary>
///
/// </summary>
ChangeSorting
};

/// <summary>
/// Used to control fetching for lazy loading
/// </summary>
[Serializable]
public class LazyLoadingCriteriaBase
{
private LoadingCommand _command;
private List<object> _idList;
private ListSortDescriptionCollection _sortInfo;

/// <summary>
/// Constructor
/// </summary>
/// <param name="command"></param>
public LazyLoadingCriteriaBase(LoadingCommand command)
{
_command = command;
}

/// <summary>
/// Only ids should be fetched
/// </summary>
public LoadingCommand Command
{
get { return _command; }
}
/// <summary>
/// Only objects with ids from the list should be loaded
/// </summary>
public List<object> IdList
{
get
{
return _idList;
}
set
{
_idList = value;
}
}
/// <summary>
/// Apply sorting when loading
/// </summary>
public ListSortDescriptionCollection SortInfo
{
get
{
return _sortInfo;
}
set
{
_sortInfo = value;
}
}
}


I believe you got the idea at this point and will not put that example. Generally, at the moment you would need to modify you SQL statements to process sorting and filter items with required primary keys.

The last step is mapping our interface to IListServer, but that's again, just trivial mapper.

One more thing. You may notice some hints of multi-threading, like locking controller, flags, etc. That part was not completed, just ignore it.

Sunday, January 11, 2009

Lazy loading for CSLA lists to support Server mode in DevExpress Grid. Part 3

Now, when we have LazyLoaderController here is what was done in ExtendedBindingList.



#region Lazy loading
private LazyLoadingController _lazyLoadingController;
private int _loadingWindowSize = 100;
private int _deletedIndex = -1;
private bool _useTheading = false;
private bool _isFetched = false;

/// <summary>
/// Change size of loading window (number of records to be loaded)
/// </summary>
/// <param name="loadingWindowSize"></param>
protected void ChangeLoadingWindowSize(int loadingWindowSize)
{
_loadingWindowSize = loadingWindowSize;
}
/// <summary>
/// Fetch data into current list
/// </summary>
/// <param name="creatria"></param>
protected virtual void FetchData(object creatria)
{
throw new Exception("FetchMoreData is not implemented");
}

protected void SetFetched()
{
_isFetched = true;
}

protected bool IsFetched
{
get { return _isFetched; }
}

/// <summary>
/// Load more records into current list if necessary to support
/// window around particular element
/// </summary>
/// <param name="index">defines what objects should be loaded (+/- window size/2)</param>
private void FetchNewWindow(int index)
{
if (_lazyLoadingController != null)
{
lock (_lazyLoadingController)
{
int startingIndex = index - (_loadingWindowSize / 2);
if (startingIndex < 0)
startingIndex = 0;
List<object> idList = _lazyLoadingController.GetUnloadedIdList(startingIndex, _loadingWindowSize);
if ((idList != null) && (idList.Count > 0))
{
LazyLoadingCriteriaBase criteria = new LazyLoadingCriteriaBase(LoadingCommand.FetchData);
criteria.IdList = idList;
FetchData(criteria);
}
}
}
}

private void FetchObject(object id)
{
List<object> idList = new List<object>(1);
idList.Add(id);
LazyLoadingCriteriaBase criteria = new LazyLoadingCriteriaBase(LoadingCommand.FetchData);
criteria.IdList = idList;
FetchData(criteria);
}
/// <summary>
/// Get unique id for child object.
/// </summary>
/// <param name="item"></param>
/// <returns></returns>
protected virtual object GetItemId(object item)
{
if (item is IIdObject)
return ((IIdObject)item).Id;
else
return item.GetHashCode();
}

protected virtual DateTime GetItemLastModified(T item)
{
if (item is IIdObject)
return ((IIdObject)item).LastModified;
else
return DateTime.MinValue;
}

/// <summary>
/// Remove Id of the object. Called internally when object removed
/// </summary>
/// <param name="id"></param>
protected void RemoveId(object id)
{
if (_lazyLoadingController != null)
lock (_lazyLoadingController)
{
_lazyLoadingController.RemoveId(id);
}
}
/// <summary>
/// Turn lazy loading on. Should be called before first fetch is done
/// </summary>
protected void SetLazyLoading(bool useTheading)
{
if (_lazyLoadingController == null)
{
_lazyLoadingController = new LazyLoadingController();
// multithreading doesn't work well yet
// _useTheading = useTheading;
}
}
/// <summary>
/// Is LazyLoading turned on
/// </summary>
public bool IsLazyLoading
{
get { return (_lazyLoadingController != null); }
}
/// <summary>
/// Some loading may be done in separate thread when lazy loading is used.
/// </summary>
protected bool UseThreading
{
get { return _useTheading; }
}

/// <summary>
/// Adds Id of the object to the list. Object is not loaded yet. Should be called
/// when ids are loaded
/// </summary>
/// <param name="id">if true, no check for existing id is done, better performance</param>
protected void AddId(object id, bool knownNew, DateTime lastModified)
{
if (_lazyLoadingController != null)
{
lock (_lazyLoadingController)
{
_lazyLoadingController.AddId(id, knownNew, lastModified);
}
}
}

/// <summary>
/// Called internally when object is loaded
/// </summary>
/// <param name="id"></param>
/// <param name="item"></param>
protected void LoadObject(object id, T item)
{
if (_lazyLoadingController != null)
lock (_lazyLoadingController)
{
_lazyLoadingController.LoadObject(id, item);
}
}

/// <summary>
/// Used internally
/// </summary>
protected void ClearIds()
{
if (_lazyLoadingController != null)
lock (_lazyLoadingController)
{
_lazyLoadingController.Clear();
}
}

/// <summary>
/// Adds object to the list of loaded objects
/// </summary>
/// <param name="index"></param>
/// <param name="item"></param>
protected override void SetItem(int index, T item)
{
base.SetItem(index, item);
object id = GetItemId(item);
LoadObject(id, item);
DateTime lastModified = GetItemLastModified(item);
if (lastModified > _lastModified)
_lastModified = lastModified;
}

/// <summary>
/// Adds object to the list of loaded objects
/// </summary>
/// <param name="index"></param>
/// <param name="item"></param>
protected override void InsertItem(int index, T item)
{
object id = GetItemId(item);
LoadObject(id, item);
DateTime lastModified = GetItemLastModified(item);
if (lastModified > _lastModified)
_lastModified = lastModified;
base.InsertItem(index, item);
}
/// <summary>
/// Need to override to call ILazyLoading.ListChanged with proper indexes
/// </summary>
/// <param name="e"></param>
protected override void OnListChanged(ListChangedEventArgs e)
{
base.OnListChanged(e);
if (_listChanged != null)
{
int newIndex;
int oldIndex;

if (_lazyLoadingController != null)
{
lock (_lazyLoadingController)
{
if (e.NewIndex < 0)
newIndex = e.NewIndex;
else
{
if (e.ListChangedType == ListChangedType.ItemDeleted)
{
if (_deletedIndex < 0)
throw new Exception("Index for ListchangeType.ItedDeleted is not defined for ILazyLoading.ListChanged");
newIndex = _deletedIndex;
_deletedIndex = -1;
}
else
{
object item = this[e.NewIndex];
object id = GetItemId(item);
newIndex = _lazyLoadingController.GetObjectIndex(id);
}
}
if (e.OldIndex < 0)
oldIndex = e.OldIndex;
else
{
if (e.OldIndex == e.NewIndex)
oldIndex = newIndex;
else
throw new Exception("ListChangedEventArgs.OldIndex not supported in for ILazyLoading.ListChanged");
}
}
}
else
{
newIndex = e.NewIndex;
oldIndex = e.OldIndex;
}
ListChangedEventArgs e2;
if (e.PropertyDescriptor != null)
e2 = new ListChangedEventArgs(e.ListChangedType, newIndex, e.PropertyDescriptor);
else
e2 = new ListChangedEventArgs(e.ListChangedType, newIndex, oldIndex);
_listChanged(this, e2);
}
}
/// <summary>
/// Load object by Id if not found. Useful only when LazyLoading is on
/// </summary>
/// <param name="id"></param>
/// <returns></returns>
protected object GetObjectyById(object id)
{
if (_lazyLoadingController != null)
{
lock (_lazyLoadingController)
{
object obj = _lazyLoadingController.GetObject(id);
if (obj == null)
{
FetchObject(id);
obj = _lazyLoadingController.GetObject(id);
}
return obj;
}
}
return null;

}

#region ILazyLoading Members
int ILazyLoading.Count
{
get
{
if (_lazyLoadingController != null)
lock (_lazyLoadingController)
{
if (!IsFetched)
{
LazyLoadingCriteriaBase criteria = new LazyLoadingCriteriaBase(LoadingCommand.LoadIds);
FetchData(criteria);
}
return (_lazyLoadingController.Count);
}
else
return Count;
}
}
object ILazyLoading.this[int index]
{
get
{
if (_lazyLoadingController != null)
{
lock (_lazyLoadingController)
{
object id;
object obj;
_lazyLoadingController.GetObjectInfo(index, out obj, out id);
if (obj == null)
{
FetchNewWindow(index);
_lazyLoadingController.GetObjectInfo(index, out obj, out id);
if (obj == null)
throw new Exception("Lazy loading didn't work, object supposed to be loaded");
}
return obj;
}
}
else
return this[index];
}
set
{
throw new Exception("The method or operation is not implemented.");
}
}
object ILazyLoading.GetRowKey(int index)
{
if (_lazyLoadingController != null)
{
lock (_lazyLoadingController)
{
object obj;
object id;
_lazyLoadingController.GetObjectInfo(index, out obj, out id);
return id;
}
}
else
{
return GetItemId(this[index]);
}
}
int ILazyLoading.GetRowIndex(object key)
{
if (_lazyLoadingController != null)
lock (_lazyLoadingController)
{
return _lazyLoadingController.GetObjectIndex(key);
}
else
{
for (int i = 0; i < Count; i++)
{
if (GetItemId(this[i]) == key)
return i;
}
return -1;
}
}
int ILazyLoading.Add(object value)
{
if (_lazyLoadingController != null)
{
lock (_lazyLoadingController)
{
((IList)this).Add(value);
object id = GetItemId(value);
return _lazyLoadingController.GetObjectIndex(id);
}
}
else
return ((IList)this).Add(value);

}
bool ILazyLoading.Contains(object value)
{
if (_lazyLoadingController != null)
{
lock (_lazyLoadingController)
{
object id = GetItemId(value);
return _lazyLoadingController.GetObjectIndex(id) >= 0;
}
}
else
return ((IList)this).Contains(value);
}
int ILazyLoading.IndexOf(object value)
{
if (_lazyLoadingController != null)
{
lock (_lazyLoadingController)
{
object id = GetItemId(value);
return _lazyLoadingController.GetObjectIndex(id);
}
}
else
return ((IList)this).IndexOf(value);
}
void ILazyLoading.Insert(int index, object value)
{
if (_lazyLoadingController != null)
{

}
else
((IList)this).Insert(index, value);
}
bool ILazyLoading.IsFixedSize
{
get
{
return ((IList)this).IsFixedSize;
}
}
bool ILazyLoading.IsReadOnly
{
get
{
return ((IList)this).IsReadOnly;
}
}
void ILazyLoading.Remove(object value)
{
((IList)this).Remove(value);
}
void ILazyLoading.ApplySort(ListSortDescriptionCollection sortInfo)
{
if (_lazyLoadingController != null)
{
lock (_lazyLoadingController)
{
LazyLoadingCriteriaBase criteria = new LazyLoadingCriteriaBase(LoadingCommand.ChangeSorting);
criteria.SortInfo = sortInfo;
FetchData(criteria);
}
}
}
private ListChangedEventHandler _listChanged;
event ListChangedEventHandler ILazyLoading.ListChanged
{
add { _listChanged += value; }
remove { _listChanged -= value; }
}
#endregion
#endregion

#region LastModified
private DateTime _lastModified = DateTime.MinValue;
/// <summary>
/// Maximum from children's LastModified
/// </summary>
public DateTime LastModified
{
get
{
if (_lazyLoadingController != null)
{
if (_lazyLoadingController.LastModified > _lastModified)
return _lazyLoadingController.LastModified;
}
return _lastModified;
}
}
#endregion




/// <summary>
/// Remove the item at the
/// specified index.
/// </summary>
/// <param name="index">
/// The zero-based index of the item
/// to remove.
/// </param>
protected override void RemoveItem(int index)
{
#region Lazy loading
object item = this[index];
object id = GetItemId(item);
if (_lazyLoadingController != null)
_deletedIndex = _lazyLoadingController.GetObjectIndex(id);
else
_deletedIndex = -1;
#endregion

OnRemovingItem(this[index]);
base.RemoveItem(index);

#region Lazy loading
RemoveId(id);
#endregion
}

Lazy loading for CSLA lists to support Server mode in DevExpress Grid. Part 2

The general idea behind this approach was fetching primary keys first and requesting objects later. My test showed that it was many times faster to fetch primary keys into the list comparing to fetching all fields and creating objects, which wasn't a surprise.

The supporting class was created that would store the mapping between primary keys and fetched objects. That was pretty simple class:



/// <summary>
/// Controls if object is loaded
/// </summary>
[Serializable]
public class LazyLoadingController
{
[Serializable]
private class ObjectInfo
{
public object _id;
public object _loadedObject = null;

public ObjectInfo(object id)
{
_id = id;
}
public ObjectInfo(object id, object loadedObject)
{
_id = id;
_loadedObject = loadedObject;
}
}
[Serializable]
private class ObjectInfoList : KeyedCollection<object, ObjectInfo>
{
protected override object GetKeyForItem(ObjectInfo item)
{
return item._id;
}
}

#region Variables
private ObjectInfoList _idList = new ObjectInfoList();
private DateTime _lastModified = DateTime.MinValue;
#endregion

#region Public methods
public DateTime LastModified
{
get { return _lastModified; }
}
/// <summary>
/// Adds Id to the list
/// </summary>
/// <param name="id"></param>
/// <returns></returns>
public void AddId(object id, bool knownNew, DateTime lastModified)
{
if (knownNew || !_idList.Contains(id))
{
_idList.Add(new ObjectInfo(id));
if (lastModified > _lastModified)
_lastModified = lastModified;
}
}
/// <summary>
/// Adds Id to the list if needed, and marks object as loaded
/// </summary>
/// <param name="id"></param>
/// <param name="loadedObject"></param>
/// <returns></returns>
public void LoadObject(object id, object loadedObject)
{
if (_idList.Contains(id))
{
ObjectInfo objInfo = _idList[id];
objInfo._loadedObject = loadedObject;
}
else
{
_idList.Add(new ObjectInfo(id, loadedObject));
DateTime lastModified = GetObjectLastModified(loadedObject);
if (lastModified > _lastModified)
_lastModified = lastModified;
}
}

private DateTime GetObjectLastModified(object obj)
{
if (obj is IIdObject)
return ((IIdObject)obj).LastModified;
else
return DateTime.MinValue;
}
/// <summary>
/// Removes id and related object from list
/// </summary>
/// <param name="id"></param>
public void RemoveId(object id)
{
_idList.Remove(id);
}
/// <summary>
/// Marks object as not loaded
/// </summary>
/// <param name="id"></param>
public void UnLoadObject(object id)
{
if (_idList.Contains(id))
_idList[id]._loadedObject = null;
}

/// <summary>
/// Gets id and object by index
/// </summary>
/// <param name="index"></param>
/// <param name="loadedObject"></param>
/// <param name="id"></param>
public void GetObjectInfo(int index, out object loadedObject, out object id)
{
IList list = _idList as IList;
ObjectInfo info = list[index] as ObjectInfo;
loadedObject = info._loadedObject;
id = info._id;
}
/// <summary>
/// Gets index and object by id
/// </summary>
/// <param name="id"></param>
public object GetObject(object id)
{
if (_idList.Contains(id))
return _idList[id]._loadedObject;
else
return null;
}
/// <summary>
/// Gets index by id
/// </summary>
/// <param name="id"></param>
/// <returns></returns>
public int GetObjectIndex(object id)
{
if (_idList.Contains(id))
{
ObjectInfo objInfo = _idList[id];
return _idList.IndexOf(objInfo);
}
else
return -1;
}
/// <summary>
/// Gets list of non-loaded object ids in particular window
/// </summary>
/// <param name="startingIndex"></param>
/// <param name="count"></param>
/// <returns></returns>
public List<object> GetUnloadedIdList(int startingIndex, int count)
{
List<object> idList = new List<object>();
if (startingIndex >= _idList.Count)
return idList;
int endIndex = startingIndex + count;
if (endIndex > _idList.Count)
endIndex = _idList.Count;
IList list = _idList as IList;
for (int i = startingIndex; i < endIndex; i++)
{
ObjectInfo info = list[i] as ObjectInfo;
if (info._loadedObject == null)
idList.Add(info._id);
}
return idList;
}
/// <summary>
/// Clears everything
/// </summary>
public void Clear()
{
_idList.Clear();
}
#endregion

#region Properties
/// <summary>
/// Number of ids in the list
/// </summary>
public int Count
{
get { return _idList.Count; }
}

#endregion

}

Lazy loading for CSLA lists to support Server mode in DevExpress Grid. Part 1

In one of our WinForms projects, I implemented so-called "lazy loading" for CSLA lists. We were very pleased with the results, so I decided to describe what was done.

First, let's state the problem I'm solving. We have classical client-server (and this is important, to make it work for n-tier, it would need to be modified) application and we want to display list of some objects, let's say customers. The list can be quite long, like 60K records. When I tried to populate that list, it took quite a long time, more than 10 seconds.

Now, we used DevExpress grid in our application that supported Server mode. To use this we would have to implement IListServer interface. The version of CSLA we used at the moment was 3.0.3

First, to avoid dependencies on user interface logic from my business layer, new interface was introduced, which is copied from IListServer.



/// <summary>
/// ILazyLoading interface is needed to support lazy loading, or loading on demand
/// </summary>
public interface ILazyLoading
{
/// <summary>
/// Number of all object in the list, including not loaded
/// </summary>
int Count { get;}

/// <summary>
/// Returns object by index. If object wasn't loaded before, it has to be loaded here
/// </summary>
/// <param name="index"></param>
/// <returns></returns>
object this[int index] { get; set;}
/// <summary>
/// Get row index by unique id
/// </summary>
/// <param name="key"></param>
/// <returns></returns>
int GetRowIndex(object key);
/// <summary>
/// Get unique id of the row by index
/// </summary>
/// <param name="index"></param>
/// <returns></returns>
object GetRowKey(int index);
/// <summary>
/// Add object to list
/// </summary>
/// <param name="value"></param>
/// <returns></returns>
int Add(object value);
/// <summary>
/// Clear
/// </summary>
void Clear();
/// <summary>
/// Contains
/// </summary>
/// <param name="value"></param>
/// <returns></returns>
bool Contains(object value);
/// <summary>
/// IndexOf
/// </summary>
/// <param name="value"></param>
/// <returns></returns>
int IndexOf(object value);
/// <summary>
/// Insert
/// </summary>
/// <param name="index"></param>
/// <param name="value"></param>
void Insert(int index, object value);
/// <summary>
/// Same as IList.IsFixedSize
/// </summary>
bool IsFixedSize { get;}
/// <summary>
/// Same as IList.IsReadOnly
/// </summary>
bool IsReadOnly { get;}
/// <summary>
/// Remove
/// </summary>
/// <param name="value"></param>
void Remove(object value);
/// <summary>
/// RemoveAt
/// </summary>
/// <param name="index"></param>
void RemoveAt(int index);
/// <summary>
/// ApplySort
/// </summary>
/// <param name="sortInfo"></param>
void ApplySort(ListSortDescriptionCollection sortInfo);
/// <summary>
/// Same as IBindingList, but need to override it to change indexes
/// </summary>
event ListChangedEventHandler ListChanged;
}


The next step is implementing this. There were two ways doing this. One is without modified CSLA library by putting implementation into my base objects. The disadvantage of this approach is that it would require repeating everything for read-only and base lists. So, I decided to not bother with that and put my implementation directly into source code, making it as less intrusive as possible. Which means I modified ExtendedBindingList little bit.

Friday, January 02, 2009

Linq to SQL performance

There is a new hype about Linq to SQL now. And honestly, I don't get it. May be I am too old :)

Anyway, I would not care about this much if it was just about how you write your code. The problem is that to save few lines of code you loose performance. I wanted to compare how much is lost when I found that someone already did it here.