Silverlight: Client-Side Database via LINQ and Isolated Storage

10. October 2008

When I did a search on "Silverlight Database" there weren't many results, and none of them actually mentioned a small database that you could embed into your Silverlight application. So, I thought I'd share a solution that I've used to store application data on the client using a combination of LINQ and Isolated Storage. This actually gives you a small "SQL"-like database since you can use LINQ to interact with it.

Basically, this technique constist of:

  1. Saving Objects in Isolated Storage as XML
  2. Keeping a copy of the data in memory to reduce disk access.
  3. Data is returned as a generic List<> object so you can use LINQ.
Also, if you use this in conjunction with web services, it allows you to cache data on the client and persist it across user session to give you a nice performance increase in certain situations.

Here's some sample code to access a table named "PersonTable" using this method:

// Load the table from the database
var people = ClientDB.PersonTable.Load();

// Select only specific data
var newPeople = (from p in people
                        where p.FirstName == "Steve"
                        select p).ToList();

// Save new data over the old
ClientDB.PersonTable.Save(newPeople);

// Here's an example of basically selecting directly
// from the table
var newPeople = (from p in ClientDB.PersonTable.Load()
                        where p.FirstName == "Steve"
                        select p).ToList();

// Remove a specific person from the table
ClientDB.PersonTable.Remove(new Guid("7687ebfc-31e6-464c-a70d-fd3bb9ecbdb4"));

Here's the code for the "PersonTable" class used above:

using System;
using System.Collections.Generic;
using System.IO.IsolatedStorage;
using System.Linq;
using System.Xml;
using System.Xml.Linq;

// Make sure to add a reference to the following in your project:
// System.Xml.Linq

namespace SimpleSilverlightDatabaseWithLINQ.Data.ClientDB
{
    public class PersonTable
    {
        private const string _filename = "PersonTable.xml";

        private static List<Person> _cache = null;
        private static object _cache_lock = new object();

        public static List<Person> Load()
        {
            // Check if data has already been loaded from
            // isolated storage, if not, then load it to memory.
            if (PersonTable._cache == null)
            {
                lock (PersonTable._cache_lock)
                {
                    if (PersonTable._cache == null)
                    {
                        var isf = IsolatedStorageFile.GetUserStoreForApplication();
                        if (!isf.FileExists(PersonTable._filename))
                        {
                            // no data has been previously saved, so set the
                            // in memory list to an empty list.
                            PersonTable._cache = new List<Person>();

                        }
                        else
                        {
                            // Data has bee previously saved, so load it.
                            var isfs = new IsolatedStorageFileStream(PersonTable._filename, System.IO.FileMode.Open, isf);

                            var reader = XmlReader.Create(isfs);
                            var doc = XDocument.Load(reader);

                            PersonTable._cache = (from e in doc.Root.Elements("Person")
                                                  select new Person
                                                  {
                                                      ID = new Guid(e.Attribute("ID").Value),
                                                      FirstName = e.Attribute("FirstName").Value,
                                                      LastName = e.Attribute("LastName").Value
                                                  }
                                                  ).ToList();

                            isfs.Close();
                            isfs.Dispose();
                        }
                        isf.Dispose();
                    }
                }
            }

            // Return the in memory list
            return PersonTable._cache;
        }

        public static void Save(List<Person> col)
        {
            lock (PersonTable._cache_lock)
            {
                // Overwrite the cache with the new list
                PersonTable._cache = col;

                // Save the list to Isolated Storage
                XElement xml = new XElement("People",
                    from p in col
                    select new XElement("Person",
                        new XAttribute("ID", p.ID.ToString()),
                        new XAttribute("FirstName", p.FirstName),
                        new XAttribute("LastName", p.LastName)
                        )
                    );

                var isf = IsolatedStorageFile.GetUserStoreForApplication();
                var isfs = new IsolatedStorageFileStream(PersonTable._filename, System.IO.FileMode.Create, isf);
                var writer = XmlWriter.Create(isfs);
                xml.Save(writer);

                writer.Close();
                isfs.Close();
                isfs.Dispose();
                isf.Dispose();
            }
        }

        public static void Remove(Guid id)
        {
            // Remove a specific Person by ID
            var people = PersonTable.Load();
            people.Remove(
                people.First(m => m.ID.ToString() == id.ToString())
                );
            PersonTable.Save(people);
        }
    }
}

And, here's the code for the Person class that is used in this example:

namespace SimpleSilverlightDatabaseWithLINQ.Data
{
    public class Person
    {
        public Person()
        {
            this.ID = Guid.NewGuid();
        }

        public Guid ID { get; set; }
        public string FirstName { get; set; }
        public string LastName { get; set; }
    }
}

Conclusion

You can actually store more data using this method than you probably think, before you begin to see performance issues. And, it could probably be optimized fairly easily to store much more data by storing the equivalent of "table index" data in memory only, and then reading from disk to retrieve the necessary records when needed. You'd just have to optimize the caching so it doesn't eat up too much memory, and you could probably store a couple hundred MB pretty easily. Anyway, "AS IS" it is perfect for storing most data that you'll need to keep track of on the client.

Silverlight , ,



Comments

10/11/2008 3:01:19 PM #
Awesome out of the box thinking!

Damon
10/13/2008 4:39:51 PM #
I have taken this pattern and used it extensively in a couple application architectures.  This works great for largely static data.  You can extend this baseline a lot further. I have added a couple big time enhancements to this pattern and I am able to perform pretty complex LINQ queries on several hundred thousand records in milliseconds.

For enterprise architecture, because Silverlight can offload simple to medium data operations this introduces almost another tier in between your service and UI layer.  Its a "true" client business layer.
11/26/2008 1:16:46 PM #
Bart Czernicki,

It would be interesting to see your changes.

I think XML is far from being the ideal format to store this data. Why don't you just simlpy serialize the collection? You could then use hash tables to be able to do indexed lookups. You can save many hash tables (this would be a pain to maintain) to create those "column" indexes. All this could be 'easily' abstracted in a Framework.... I'd better go code it and post something.... I might be back in a couple of days (if I made it work at all)
11/26/2008 1:17:15 PM #
What's wrong with my avatar? Where did it come from?
11/26/2008 1:51:27 PM #
What do you mean serialize the collection?  The collections are on the client and you need those hash indexes as helpers with joins.  So for example filter/join patterns work like this:

if (hash1 = hash2) // which is really quick since hashes find values quicker big o notation efficient
  check if the value matches // use the int or string id to ensure it matches
else // no match

If you add additional threads to this index/filter lookup you get more power Smile
11/27/2008 9:09:13 AM #
I know the collections are on the client. What I mean is to save the collection directly instead of having to convert it to XML...
11/27/2008 9:24:10 AM #
I see what you are saying.  I never used XML storage, I think we got each other confused.  I actually pass objects with List initializers with hash tables to manage finds/filters/joins.  So, what happens is that I pass a XAP file and it has my pre-compiled data and then I load that into memory and do my work.  If you are familiar with Business Intelligence, its very analogous to passing around MOLAP cubes (Cogons Cubes or in Analysis Services .cub files).

Microsoft is extending this pattern in SQL Server 2008 R2, which they demoed at their BI conference where they had a cube file with aggregates (i.e. smart data structures) able to do rapid and real-time filtering on millions of rows.  Add multi-threading to do this and a couple more advanced probability algorithms from statistics and you can replicate this in Silverlight (which I did with several hundred thousand records).

Back to my point, without hash indexes this thing is 2-3x slower!
11/27/2008 1:02:22 PM #
That's why I tought about hash indexes. I don't know your implementation, but I mentioned the XML storage because of the article. That's the proposed implementation.
12/27/2008 6:08:27 PM #
When I did a search on "Silverlight Database" there some niche results
at the moment. what it means? donno what the move makes
Comments are closed