Hibernating Rhinos

Zero friction databases

Hibernating Rhino is featured in DZone’s latest research guide

Growing Database Trends: NoSQL and Polyglot Persistence

Hibernating Rhino is featured in DZone’s latest research guide, the Guide to Database and Persistence Management. This guide features articles from industry experts, key findings from a survey of over 800 developers, an infographic differentiating between databases, and a solution directory of the leaders in database and persistence management. You can download the free research guide here: http://bit.ly/1BmCMbb

As the amount of data generated grows, companies are facing new and unprecedented data management challenges. It’s estimated that the amount of data we currently have will double every two years -- making these challenges more difficult to overcome. DZone’s Guide to Database and Persistence Management is a valuable handbook for understanding and conquering the challenges posed by modern database usage.

Based on DZone’s survey, performance, scalability and data migration are the biggest challenges related to databases.

DZone’s research guide also reports that 37% of respondents expect that their data management technologies will not be sufficient by next year, and they expect to add a new database product to their software by 2016.

 

NoSQL Databases on the Rise

While SQL databases have dominated the IT industry, experts are suggesting that organizations use a multi-database solution or polyglot persistence approach. This gives your organization more flexibility for organizing various types of data, while allowing you to handle each type of data with a more efficient database.

DZone reports that 30% of respondents’ organizations currently use a polyglot persistence approach with at least one relational database and one non-relational database. They also reported that 80% of respondents used two or more database products in their organization.

Read DZone’s Guide to Database and Persistence Management to navigate the multitude of persistence options, while discussing the challenges of analyzing data in organizations.

Tags:

Published at

Originally posted at

Spatial Sorting

Hi all,

I’d like to share with you a new feature we were working on in the sphere of spatial queries.

It began from an issue, telling that we need to have the ability to sort query results by distance from a given point, without limiting the scope of the query. Truth to be told, it took me some time to understand what the problem was and why are the results were limited at the first place. To explain that point, I'll give an example of how spatial queries looks like in the Client API:

Let’s say that our document looks like:

public class Shop { public string Name { get; set; } public double Lat { get; set; } public double Lng { get; set; } }

And our index looks like:

public class ShopSpatialIndex : AbstractIndexCreationTask<Shop> { public ShopSpatialIndex() { Map = shops => from shop in shops select new { shop.Name, _ = SpatialGenerate(shop.Lat,shop.Lng) }; } }

Now, there are various ways to perform a spatial query, somehow I stumbled upon the most detailed and complex one:

store.DatabaseCommands.Query("ShopSpatialIndex", new SpatialIndexQuery() { QueryShape = SpatialIndexQuery.GetQueryShapeFromLatLon(36.15632, 51.70375, 100), SpatialRelation = SpatialRelation.Within, SpatialFieldName = Constants.DefaultSpatialFieldName, SortedFields = new[] { new SortedField(Constants.DistanceFieldName), } });

In that case, you can see that we are allowed to mention a shape, the relation the shape should have to the data, queried field and sorted field. The problem is that the sorting is done according to the center of the given field.

After looking a bit more, I found out that there is a special kind of relation: “Nearby”,  that one allows to do a sorting, without filtering:

store.DatabaseCommands.Query("ShopSpatialIndex", new SpatialIndexQuery() { QueryShape = SpatialIndexQuery.GetQueryShapeFromLatLon(36.15632, 51.70375, 100), SpatialRelation = SpatialRelation.Nearby, SpatialFieldName = Constants.DefaultSpatialFieldName, SortedFields = new[] { new SortedField(Constants.DistanceFieldName), } });

Now I  had a new problem: I cannot do both. Example for case when I'd like to do that is if I'm searching for Shops in a particular city, I stand inside or outside it and I want the results to be ordered by proximity:

 

image

Although the “nearby” relation inspired me, I couldn’t use it in order to solve the problem. What I chose to do is to allow passing the center of the sorting with the sorted field name, in manner of CSV parameter concatenation, no worry, I encapsulated it into a nice API. Although this solution made the complex, and most detailed writing of the query to look even less intuitive:

store.DatabaseCommands.Query("ShopSpatialIndex", new SpatialIndexQuery() { QueryShape = SpatialIndexQuery.GetQueryShapeFromLatLon(36.15632, 51.70375, 100), SpatialRelation = SpatialRelation.Nearby, SpatialFieldName = Constants.DefaultSpatialFieldName, SortedFields = new[] { new SortedField(string.Format("{0};{1};{2}", Constants.DistanceFieldName, 36.15612, 51.70355)), } });

The alternative methods to perform spatial querying now looks like:

session.Query<Shop>() .Customize(x => x.WithinRadiusOf(100, 36.15632, 51.70375)) .OrderByDistance(new SpatialSort { Latitude = 36.15612, Longitude = 51.70355, FieldName = Constants.DefaultSpatialFieldName });

or

session.Query<Shop>() .Customize(x => x.WithinRadiusOf(100, 36.15632, 51.70375)) .Customize(x => x.SortByDistance(100, 36.15632, Constants.DefaultSpatialFieldName))

or

session.Query<Shop>() .Customize(x => x.WithinRadiusOf(100, 36.15632, 51.70375)) .Customize(x => x.SortByDistance(36.15612, 51.70355))

Hope you find it useful and it will encourage you to explore and use this and other spatial functionality of RavenDB.

Tags:

Published at

Originally posted at

Test Suite for RavenDB - rough ideas

By Jakub Rusilko

For the last couple of weeks we have been trying to come up with a good way of assuring better stability of RavenDB. It turns out that manual tests and classical unit tests are not enough. Thus we figured that a complex test suite is needed.

We have distinguished a few separate things that we would like to be able to test. The first is the client API which can be broken into two categories:

  1. the .NET API
  2. the REST API

Within the .NET API we can further distinguish some different aspects that we would like to cover:

  1. sync vs async
  2. remote vs embeded
  3. self host vs IIS

Apart from that we'll also create tests for the external tools:

  1. Backup
  2. Smuggler.

These will include migration tests.

In every test we'll try to cover the correctness of the results as well as the execution times. We are also planning to be able to run stress tests in single and multiple threads.

What we are currently trying to design is the way to be able to easily test different versions of RavenDB so that we can compare things across versions. Our ultimate goal is to be able to collect all the described above test data and later produce meaningful charts or graphs based on the data to help in tweaking the code. We hope that the results will not only show us whether something works or not in a given version but also how the execution time of specific operations change in different database versions.

At this point it is difficult to be more specific about the project. We are still just trying out different approaches to see what will work best for us.

Tags:

Published at

Originally posted at

Comments (1)

New Client Coming to RavenDB – Mono For Android

You requested it and we listened, we now have a client for Android (with mono).

The client supports all the regular client side operation except from Distributed Transactions which are not relevant for mobile devices

The Client will be available through a Nuget Package or by referencing the dll to your project

All you need to do is reference the Raven.Client.MonoForAndoird.dll to your Mono for android project.

In there you can use the code as you normally would.

If you want to start working with this already you can download the dll here:

https://www.dropbox.com/s/e2n5fioobv3v09u/Raven.Client.MonoForAndroid.dll

Pay attention this is an early build, so if you have something not working, we will appreciate if you could create a failing test for it

You can see how we are testing from this repository https://github.com/DanielDar/ravendb on branch 2.5

Look at the project Raven.Tests.MonoForAndroid

There is also Raven.Tryouts.MonoForAndroid for testing a complete app

You can see a sample for the using android with raven here: https://github.com/DanielDar/RavenAndroidToDoList

In this sample you have a to do list that is updated between all of the android devices running it.

Tags:

Published at

Originally posted at

Comments (1)

New feature in RavenDB Studio: get spatial geocode from address

For a while now we had the option to do spatial search, in order to do that we need the longitude and latitude of center we want to search around.

However most of the time that we really want to check is result round a specific location (we don’t really care about the longitude and latitude but the address they represent) so usually you would look in some site that you input an address and you get the longitude and latitude.

With the new feature in RavenDB Studio we let you enter an address and we update the longitude and latitude for you:

The new UI:

clip_image002

We enter an address:

clip_image004

And after we press the "Calculate from address":

clip_image006

It is that simple.

In my next post, I’ll discuss exactly how we did it. In turned out that it isn’t as trivial in Silverlight as we might want.

Published at

Originally posted at

Dealing with conflicts in RavenDB studio

Recently we added the option to manage bundles in the studio (see http://blogs.hibernatingrhinos.com/12577/ravendb-1-2-studio-features-the-new-database-wizard).

One of these bundles is the Replication bundles, sometimes with replication a conflict can happen (this could happen by several reason like 2 or more documents with the same id being created on different destinations), the way RavenDB deals with this is by creating one document to each of the versions available of the document and in the “original” document we have the names of these copies.

With the new conflict resolution UI when you try to edit the original document you will receive a suggestion of resolution.

Let create a conflict on purpose and see some of the options:

In order to create the conflict we will create 2 databases on the same server, both with replication bundle enabled but we will not set the destination yet:

clip_image002

Now we will create a document on each database with the same Id:

 

clip_image004 clip_image006

Now we will edit the destinations on the left and connect it to the right :

clip_image008

Now we will go to the destination and look the the documents list:

clip_image010

As you can see we have 3 documents with Id starts with “Example/1”.

Now we get to the interesting part if we go and try to edit “Example/1” we will get the next page:

clip_image012

The Studio realizes that we wanted to edit a document that had a conflict and gave us a suggestion, as long as there are comments on the document you will not be able to save it.

Now you can see that RavenDB Studio saw an issue with the Name, we give you a list of all possible names and all you need to do is delete what you don’t want there.

Let say we want it to be “Hibernating” we will delete what we don’t want and save:

Before save:

clip_image014

After save:

clip_image016

Now if we take a look at the documents list you can see that we are left with just the fixed version:

clip_image018

Now let look what RavenDB will suggest for several other conflicts:

Example 2 – Same Item:

clip_image020 clip_image021

The result:

clip_image023

Notice that we still let you know that a conflict has accrued but you have not comments in there which means you can simply save to resolve the conflict

Example 3 – List With Different Items

clip_image025 clip_image027

The result:

clip_image029

We put both items in the array but add the comment so that the user will know that the data was changed.

Example 4 – List with some items the same and some different

clip_image031 clip_image033

The result:

clip_image035

We combined all the data into one array with no repeats and inform on the change.

For more about conflicts read: https://ravendb.net/docs/server/scaling-out/replication/handling-conflicts

Creating a Wizard in Silverlight

When creating the new “Create New Database” for the studio we ran into an issue with showing several child windows one after the other (and keeping in mind that if in one page cancel was click we don’t want to show the next windows) like an install wizard.

Unfortunately Silverlight does not have anything like that so we needed to create on for ourselves.

First thing we had in mind when we came to approach this is that Silverlight works asynchronously which means that we want to wait for a page to close before we open the next one.

The way we dealt with that was listening to every windows Closed event and on close open the next page (if exists) the Wizard class looks like this:

clip_image002

The ShowNext() method will check to see if we have more windows to show and show the next one or return with the result if we are done.

We use a TaskCompletionSource and the TPL to make sure that all of that is wrapped in a nice Task, and that is pretty much it. With that done, we can start using it. You can see how easy it is here:

clip_image004

Published at

Originally posted at

Dynamic Tab View In Silverlight

With the new “Create New Database” feature to the RanvenDB Studio we added the option to edit your bundles.

We wanted something that seemed to be quite strait forward: A tab view that we could dynamically choose which tabs a visible and which are not.

At first we tried doing it with the TabControl and bind the header visibility of each TabItem to a property in the model that let us know if this database has that bundle.

This is how I had done it:

clip_image002

At first it seemed to work, however when looking closely we noticed something like this:

clip_image004

clip_image006

As you can see, in the header we only have “Versioning” but the tab we see is the “Quotas Tab”

This happened because we hid the header but the item itself was still the “SelectedItem”.

So we tried to bind the “SelectedItem” as well, but apperantly there is an issue with TabControl and MVVM.

Finally we decided to create our own implementation of Tab View and this is how we did it:

clip_image008

In the Model size we have:

clip_image010

The Bundles Collection is updated with the list of bundles that are active for this database.

One other this we needed to add was to initialize the SelectedBundle to be the first one in the list.
We did that where we initialize the Bundles collection like that:

clip_image012

RavenDB 1.2 Studio Features: The New Database Wizard

In the new version of the RavenDB Management Studio we wanted to enhance the options you have, one of these enhancements is the new "Create New Database" wizard.

We wanted to add the option to choose bundles for your database, configure various options and in general give you a easy way to control options that you could previously do only if you mastered the zen of RavenDB.

Now then you select the New Database option from the databases page you will get the next window:

clip_image002

If you check the “Advanced Settings” option you will get this window:

clip_image004

Now you can change the location where the data (path), logs and indexes will be stored.
In some cases putting the data, logs and the indexes in different drives can improve performance.

On the left side you have the Bundles Selection area, every bundle that you select here will be added to your database. More data about bundles: https://ravendb.net/docs/server/bundles

Some of the bundles have settings that you can (and should) configure:

Encryption:

clip_image006

In here you can choose you encryption key (or use the random generated on) after the database creation a pop up with the key will show you the key on last time. Make sure you have this key saved somewhere; after that window will close we will not give you the key again.

Quotas Bundle

clip_image008

In here you can set the limits for quotas (see http://ravendb.net/docs/server/bundles/quotas)

Replication Bundle

clip_image010

In here you can set the replications (and add as many as needed)
(see http://ravendb.net/docs/server/scaling-out/replication)

Versioning

clip_image012

In here you can set the versioning (see http://ravendb.net/docs/server/bundles/versioning)

Notice that you can’t remove the default configuration or change the Collection for it but you can edit the values.

After you will press “OK” the database will be created (if you have the encryption bundles you will now see the encryption key one last time)

If in the future you want to edit the setting to the bundles (for quotas, replication and versioning) you can right-click the database in the databases view and select “edit bundles”

clip_image014

Note: you cannot change which bundles are part of the database after creation.

Published at

Originally posted at

Implementing a VirtualizingWrapPanel

This is a guest post by Samuel Jack, a freelancer who has worked with us on the new RavenDb Studio features.

If you’ve used Silverlight or WPF for any length of time, you will know all about Panels – StackPanel, DockPanel, Grid and the like - which handle the positioning of UI elements on screen. You’ll also know about VirtualizingStackPanel, which can work wonders for UI performance when displaying substantial lists of data. It does this by deferring the creation of UI elements for off-screen list items until they are scrolled into view. (If you don’t know about Panels, and want to learn, a good place to start is on Dr WPF’s blog)

What if you have a huge stash of data but don’t want to display it in a boring old list? What if you want items laid out like cards on a table? Silverlight and WPF both include a WrapPanel class which can achieve the desired layout. But as its name suggests, it doesn’t virtualize. Use a WrapPanel in your 10,000-item ListBox, and you’ll be twiddling your thumbs whilst it goes off to create elements you might never see.

This was a problem we faced when we were implementing the new data virtualization features in RavenDb Studio 1.2, which lets you scroll through your entire collection of documents. We wanted the card layout, but WrapPanel wasn’t up to the job of working with huge numbers of items.

So I set about implementing a VirtualizingWrapPanel, which I’ll share with you today. You can see it in action in our sample Netflix browser (switch the Display Style to Card). The code is all available on GitHub.

First I’ll give you a quick refresher on how you make ListBox use a custom Panel, then I’ll run through how one goes about implementing a virtualizing panel in Silverlight.

A Quick Refresher on Using Panels

Here’s a snippet of XAML showing a ListBox using a VirtualizingWrapPanel:

<ListBox x:Name="DocumentsList"
                        ItemsSource="{Binding Items}" DisplayMember="Item.Name">
      <ListBox.Template>
          <ControlTemplate>
              <Border CornerRadius="2" 
                    BorderBrush="{TemplateBinding BorderBrush}"
                    BorderThickness="{TemplateBinding BorderThickness}">
                  <!-- We set the TabNavigation to Cycle on this ScrollViewer to work around a bug which causes the ListBox to loose focus when navigating down (with Down Arrow or Page Down) from the last visible item
              (or even when navigating Up/Down on an item that is only partially visible at the bottom of the screen) -->
                  <ScrollViewer x:Name="ScrollViewer" Padding="{TemplateBinding Padding}" Background="{TemplateBinding Background}" BorderBrush="Transparent" BorderThickness="0" 
                                  TabNavigation="Cycle" IsTabStop="False">
                      <ItemsPresenter />
                  </ScrollViewer>
              </Border>
          </ControlTemplate>
      </ListBox.Template>
    <ListBox.ItemsPanel>
        <ItemsPanelTemplate>
            <VirtualCollection1:VirtualizingWrapPanel ItemWidth="200"
                                            ItemHeight="230"/>
        </ItemsPanelTemplate>
    </ListBox.ItemsPanel>
</ListBox>
The key part is right down at the bottom of the snippet, where we set the ItemsPanel property of the ListBox. There we specify an ItemsPanelTemplate which instantiates the VirtualizingWrapPanel. Notice that you need to tell the panel how big each item is going to be up front, so that it knows how much room to allow for items which are off screen.

There’s one other important point. Notice that I’ve overridden the Template property for the ListBox. This is so that I can tweak the ScrollViewer which is going to contain the VirtualizingWrapPanel. The tweak is a small one: I set its TabNavigation property to Cycle. Without this, keyboard navigation doesn’t work properly within the VirtualizingWrapPanel: under certain circumstances, navigating with the Up/Down or Page Up/Page Down keys would move the keyboard focus out of the ListBox, rather than to the next item in the ListBox.

Birds-Eye View of a Virtualizing Panel

So how do you implement a Virtualizing Panel in Silverlight?

There’s a post on my own blog which explains how to create Panels of the standard variety, and I suggest you go there first for a bit of background. Suffice to say here that normal Panels have one job in life: they must layout the elements given to them by their parent control.

Virtualizing Panels have two additional responsibilities. First, they must keep track of their own scroll state. And second, they must manage the creation of UI elements - item containers -for each data item as and when they are needed. They don’t actually create the elements – that is done by an ItemContainerGenerator. They just decide when each item container should be generated.

Implementing a virtualizing panel that can handle UI items of any size you throw at it is a very tricky job. That’s because, in order to accurately position any given element, you need to know the sizes of all the elements that come before it in the list. To know their sizes, you have to measure them, and to be measured, they need to exist. But the very reason for using a virtualizing panel is to avoid creating item unnecessarily. That is why virtualizing panels tend to take one of two easy routes out of the conundrum.

VirtualizingStackPanel’s favoured approach is to use index based scrolling, where it only ever shows whole items, and the scroll bar reflects the current items’ indexes in the list and has nothing to do with their sizes. This has the downside of jerky scrolling if you have large items, since each movement of the scrollbar will move previous items entirely out of view.

Jerky scrolling is avoided by the approach I’ve taken, which is to require that each item displayed by the panel is the same size. This enables us to easily calculate the position of any given item based on its index in the list.

Scrolling Responsibilities

How does a virtualizing panel discharge its scrolling responsibilities?

The panel needs to implement IScrollInfo so its containing ScrollViewer knows that it is going to manage its own scrolling. IScrollInfo’s job splits into two parts: one is to report on the current scroll state through properties like ExtentHeight and ViewportWidth. The Extent represents the logical size of the panel – what it would be if no scrolling was taking place and everything was visible at once. The Viewport is the rectangle of the panel that is actually visible at this moment.

IScrollInfo’s other job is to enable that scroll state to be manipulated, through methods like LineUp and MouseWheelDown. In many implementations, these manipulation methods all end up by calling one of two other methods on IScrollInfo, SetVerticalOffset and SetHorizontalOffset.

Both SetVerticalOffset and SetHorizontalOffset do much the same thing, so I’ll just show the SetVerticalOffset method:

public void SetVerticalOffset(double offset)
{
  if (_isInMeasure)
  {
      return;
  }

  offset = Clamp(offset, 0, ExtentHeight - ViewportHeight);
  _offset = new Point(_offset.X, offset);

  InvalidateScrollInfo();
  InvalidateMeasure();
}

After checking that the given offset is within the scrollable range, the method simply updates a field to show where the panel is scrolled to. Then it notifies its parent ScrollViewer that its state has changed, and, by calling InvalidateMeasure, tells the framework that its layout (i.e. the arrangement of its children) needs updating. The method does one other check: if it discovers that the panel is already in the middle of updating its layout (i.e. _isInMeasure is true) it doesn’t bother changing the scroll offset. This circumvents a problem I’ve encountered where Silverlight tries to scroll to elements newly created during the layout process.

Handling Layout

Layout happens in two parts. First we measure, calculating how much space the panel will occupy taking into account its children; then we arrange, allocating each child its place within the parent. Measuring happens in MeasureOverride, and arranging in ArrangeOverride. Most of the work in the VirtualizingWrapPanel actually happens in MeasureOverride because the panel needs to generate items containers before it can measure them. You might wonder why we need to measure the item containers when we’ve already decided that they have a fixed size? That is so that the item containers can handle their own layout, which happens using the same Measure and Arrange process.

Before I show you the code for MeasureOverride, here’s an outline of what it does:

  1. GetExtentInfo is called to calculate the full height of the list – i.e. how tall it would be if we displayed it all instead of scrolling.
  2. GetLayoutInfo then works out, based on the scroll offset, which item indexes fall inside the visible range. We always make sure to include realize one item before the first visible item, and one item after the last visible item. This enables keyboard scrolling to work properly, because Silverlight can then scroll that item into view, thus triggering the panel to realize the rest of the row.
  3. Any of the current item containers that now fall outside of the visible range are recycled – handed back to the ItemContainerGenerator so that it can reuse them for other items.
  4. MeasureOverride then loops through the visible item indexes and:
    1. Asks the ItemContainerGenerator to create the UI container for the item. If the visibility of the item hasn’t changed, ItemContainerGenerator will give us back the same item container we had before
    2. Calls SetVirtualItemIndex to tag the container with the index in the list that it is now representing. This is used in step 3 the next time round when we check for items that fall outside of the visible range.
    3. Makes sure the container is in the correct place in the Panel’s list of visual children, and thus part of the visual tree. If an item is not in the correct order, keyboard navigation won’t work correctly
    4. Asks the ItemContainerGenerator to Prepare the container – apply its styles, etc.
    5. Measures the item container – this must come after step 4, because until that point the item container isn’t fully initialized, and might not even know what its content is.
    6. Decide where exactly the item container is going to be displayed: its layout rectangle is recorded in the _childLayouts dictionary. This is used by the ArrangeOverride method to arrange each child
  5. Now we can go through the panel’s list of item containers and remove any which were never actually reused. Back in step 3 every existing item container’s VirtualItemIndex tag was set to –1; and in step 4.2, containers which are in use have their tag set to the item of the index they are now representing. So item containers which were never reused will still have a VirtualItemIndex of –1.
  6. Update the scroll state with the new extent height and viewport size so that scroll bars can be drawn in the correct proportions.
  7. Decide what size to report as the desired size for the Panel. The Panel wants to take up all the available space (within the confines of its parent), so in most cases, that’s what it reports. But if it is told that the available size is Infinite, then it returns a size of 0, and by convention, Silverlight takes that to mean “use all the available space”.

Here’s the code:

protected override Size MeasureOverride(Size availableSize)
{
  if (_itemsControl == null)
  {
      return availableSize;
  }
  
  _isInMeasure = true;
  _childLayouts.Clear();

  var extentInfo = GetExtentInfo(availableSize, ItemHeight);

  EnsureScrollOffsetIsWithinConstrains(extentInfo);

  var layoutInfo = GetLayoutInfo(availableSize, ItemHeight, extentInfo);

  RecycleItems(layoutInfo);

  // Determine where the first item is in relation to previously realized items
  var generatorStartPosition = _itemsGenerator.GeneratorPositionFromIndex(layoutInfo.FirstRealizedItemIndex);

  var visualIndex = 0;

  var currentX = layoutInfo.FirstRealizedItemLeft;
  var currentY = layoutInfo.FirstRealizedLineTop;

  using (_itemsGenerator.StartAt(generatorStartPosition, GeneratorDirection.Forward, true))
  {
      for (var itemIndex = layoutInfo.FirstRealizedItemIndex; itemIndex <= layoutInfo.LastRealizedItemIndex; itemIndex++, visualIndex++)
      {
          bool newlyRealized;

          var child = (UIElement)_itemsGenerator.GenerateNext(out newlyRealized);
          SetVirtualItemIndex(child, itemIndex);

          if (newlyRealized)
          {
              InsertInternalChild(visualIndex, child);
          }
          else
          {
              // check if item needs to be moved into a new position in the Children collection
              if (visualIndex < Children.Count)
              {
                  if (Children[visualIndex] != child)
                  {
                      var childCurrentIndex = Children.IndexOf(child);

                      if (childCurrentIndex >= 0)
                      {
                          RemoveInternalChildRange(childCurrentIndex, 1);
                      }

                      InsertInternalChild(visualIndex, child);
                  }
              }
              else
              {
                  // we know that the child can't already be in the children collection
                  // because we've been inserting children in correct visualIndex order,
                  // and this child has a visualIndex greater than the Children.Count
                  AddInternalChild(child);
              }
          }

          // only prepare the item once it has been added to the visual tree
          _itemsGenerator.PrepareItemContainer(child);

          child.Measure(new Size(ItemWidth, ItemHeight));

          _childLayouts.Add(child, new Rect(currentX, currentY, ItemWidth, ItemHeight));

          if (currentX + ItemWidth * 2 >= availableSize.Width)
          {
              // wrap to a new line
              currentY += ItemHeight;
              currentX = 0;
          }
          else
          {
              currentX += ItemWidth;
          }
      }
  }

  RemoveRedundantChildren();
  UpdateScrollInfo(availableSize, extentInfo);

  var desiredSize = new Size(double.IsInfinity(availableSize.Width) ? 0 : availableSize.Width,
                             double.IsInfinity(availableSize.Height) ? 0 : availableSize.Height);

  _isInMeasure = false;

  return desiredSize;
}

It’s a Wrap

That concludes our look at the VirtualizingWrapPanel. Go and check out the whole sample on GitHub, and let us know what you make of it.

Data Virtualization in Silverlight: Digging into VirtualCollection

This is a guest post by Samuel Jack, a freelancer who has worked with us on the new RavenDb Studio features.

The other day I showed you how we enabled the display of massive scrollable lists of items in Silverlight by implementing data virtualization for DataGrids and ListBoxes. If you’ve looked at that sample, you might be curious as to how it all works.

That is the subject for todays post.

What is Virtualization?

If you’re familiar with Silverlight or WPF, you’ll know all about virtualization when it comes to ListBoxes and DataGrids. Virtualization is a trick the framework uses to avoid having to render UI for items in a list that are not visible on screen. It creates UI for items that are visible, and as the user scrolls, it destroys the UI elements for data that has disappeared off screen, and creates new elements for items that have just come into view. VirtualizingStackPanel is one of the classes in WPF and Silverlight that is responsible for handling this on behalf of controls like the ListBox.

VirtualCollection, the class that lies at the heart of our solution, takes this one step further, and applies the same idea to the items in the list themselves. Rather than loading all the items into memory up front, it pulls in just the ones that are needed as the user scrolls (which might involve making a call to a server), and unloads any items which are not likely to be needed again any time soon. Sounds simple enough in theory, but getting this to work in Silverlight involves a bit of trickery.

Appetite reduction therapy, Silverlight style

The first problem we come up against is Silverlight’s I-want-it-and-I want-it-all-NOW attitude to data. If you present a Silverlight ListBox or DataGrid with a simple IList to be its data source, Silverlight will ask the list for its Count, and then proceed to request every single item in the list, irrespective of how much it can actually display at that moment. This isn’t a problem in most use cases, but when we want our VirtualCollection to display lists with potentially billions of data items, that approach just isn’t tenable. So we have to find a way to get Silverlight to slim down its appetite for data.

The trick here, it turns out, is to implement ICollectionView as well as IList. Flaunting this extra interface seems to be enough to persuade Silverlight to back down, and just ask for the items it can actually display, as it needs them. There is some extra work involved, as ICollectionViews are supposed to be responsible for keeping track of the current item selected in a list, but we can handle that.

So with ICollectionView implemented, Silverlight now asks the VirtualCollection for items it wants to display one by one. But that leads to the next problem. When a ListBox, say, first comes knocking on  the door,  VirtualCollection doesn’t have any items in stock. They’re all on the server, and have to be fetched into memory. But VirtualCollection can’t keep Silverlight waiting whilst it goes and does that, or Silverlight will sulk and hang the browser. So VirtualCollection fobs Silverlight off by giving it back a VirtualItem which acts as a place-holder for the real item. Silverlight can then happily create a row in the ListBox, albeit a blank row, because it has no actual data yet. When the real item arrives, fresh from the server, VirtualCollection hands it to the VirtualItem, and through the magic of data binding, the row lights up with the appropriate data.

Let’s take a look at some code, to see how VirtualCollection implements this process.

VirtualItems

We’ll start where Silverlight starts, when it wants to display a particular item: at VirtualCollection’s indexer:

public VirtualItem<T> this[int index]
{
  get
  {
      RealizeItemRequested(index);
      return _virtualItems[index] ?? (_virtualItems[index] = new VirtualItem<T>(this, index));
  }
  set { throw new NotImplementedException(); }
}

Two things are happening here: first up, we’re kicking off a request to realize the item – fetch it from the server, in other words. Secondly, we’re creating a VirtualItem to stand in for the real item – or if we’ve already created a VirtualItem representing that index, we simply return that.

It’s worth saying a word about how we store all these VirtualItems, because they are not stored in a standard List. The reason for that is that there could be billions of them, and we wouldn’t want to allocate an array that big (which is what a List would do). That would be especially wasteful given that the vast majority of the array would never be used, since the user is unlikely to look at every single item.

So VirtualItems are stored in a SparseList, which allows items to be accessed by index as conveniently as if they  were in one big array. Under the covers it actually stores items in lots of smaller arrays which can be more easily garbage-collected when they’re no longer needed.

To Fetch a Page of Data

Let’s chase the process along a little further.

public void RealizeItemRequested(int index)
{
  var page = index / _pageSize;
  BeginGetPage(page);
}

private void BeginGetPage(int page)
{
  if (IsPageAlreadyRequested(page))
  {
      return;
  }

  _mostRecentlyRequestedPages.Add(page);
  _requestedPages.Add(page);

  _pendingPageRequests.Push(new PageRequest(page, _state));

  ProcessPageRequests();
}

Items are never fetched from the server as individuals, but always in pages, so that we can make the most of each round-trip across the wire. The first thing that RealizeItemRequested does, therefore, is work out which page the requested index is part of. Then it puts in a request to fetch that page.

BeginGetPage doesn’t do any fetching directly. After ensuring it isn’t wasting time fetching a page it already has, what it does do is push a fetch-request for the page on to a stack, _pendingPageRequests. It also adds the page to a list it maintains of the mostly recently requested pages. This list is used to manage the cache of pages which are kept in memory. When the list gets too long, the oldest pages are evicted from memory.

Finally BeginGetPage goes off to process the page requests.

private void ProcessPageRequests()
{
  while (_inProcessPageRequests < MaxConcurrentPageRequests && _pendingPageRequests.Count > 0)
  {
      var request = _pendingPageRequests.Pop();

      // if we encounter a requested posted for an early collection state,
      // we can ignore it, and all that came before it
      if (_state != request.StateWhenRequested)
      {
          _pendingPageRequests.Clear();
          break;
      }

      // check that the page is still requested (the user might have scrolled, causing the 
      // page to be ejected from the cache
      if (!_requestedPages.Contains(request.Page))
      {
          break;
      }

      _inProcessPageRequests++;

      _source.GetPageAsync(request.Page * _pageSize, _pageSize, _sortDescriptions).ContinueWith(
          t =>
          {
              if (!t.IsFaulted)
              {
                  UpdatePage(request.Page, t.Result, request.StateWhenRequested);
              }
              else
              {
                  MarkPageAsError(request.Page, request.StateWhenRequested);
              }

              // fire off any further requests
              _inProcessPageRequests--;
              ProcessPageRequests();
          },
          _synchronizationContextScheduler);
  }
}

There are a number of interesting design decisions revealed by ProcessPageRequests, so it’s worth looking at in some detail.

First, why do we use a stack for managing the pending page requests, and not, say, a queue? Well, think about what happens when a user scrolls a long way through a list, and then stops. As they scroll, page requests will be generated, and will take some time to process, so the queue would grow. By the time the user stops scrolling, the queue might be quite long, and it would take a little time for the most recent requests, the ones for the pages the user is now staring at, to be dealt with. By putting the requests in a stack, which is a first-in-first-out data structure, we ensure that the most recent requests, those most likely to be relevant, are dealt with first.

We might also find that there are some requests which are no longer relevant. If the user scrolled a very long way, causing lots of pages to be added to the _mostRecentlyRequestedPages list, there may well be some pages which have to be kicked out of the cache. Any fetch requests for those pages can be safely ignored.

Requests can become irrelevant for another reason too: the collection state may have moved on since the request was made. The collection state is a concept that helps to make sure the VirtualCollection does the right thing at the right time in spite of all the asynchronous activity going on. The state is simply a number which is incremented every time VirtualCollection is notified by the IVirtualCollectionSource that the collection has changed (this is done in UpdateData and Reset). Whenever an asynchronous action is started, say a request to fetch data from the server, VirtualCollection tags the request with the state at that instant. When the response of the asynchronous action comes back, VirtualCollection compares its current state with the state when the action was begun. If the two are different, VirtualCollection knows that the world has moved on, so that response is no longer relevant.

Notice, finally, in this method that when we fire off the asynchronous request to get a page of data – this is where the IVirtualCollectionSource comes into play – we make sure the response is handled in the UI thread;  that is the purpose of passing _synchronizationContextScheduler to ContinueWith. All responses to asynchronous requests are handled in this way to make sure we don’t have any race conditions when updating VirtualCollection’s internal state.

Acting on the Data

What happens when the data is received from the server?

private void UpdatePage(int page, IList<T> results, uint stateWhenRequested)
{
  if (stateWhenRequested != _state)
  {
      // this request may contain out-of-date data, so ignore it
      return;
  }

  bool stillRelevant = _requestedPages.Remove(page);
  if (!stillRelevant)
  {
      return;
  }

  _fetchedPages.Add(page);

  var startIndex = page * _pageSize;

  for (int i = 0; i < results.Count; i++)
  {
      var index = startIndex + i;
      var virtualItem = _virtualItems[index] ?? (_virtualItems[index] = new VirtualItem<T>(this, index));
      if (virtualItem.Item == null || results[i] == null || !_equalityComparer.Equals(virtualItem.Item, results[i]))
      {
          virtualItem.SupplyValue(results[i]);
      }
  }

  if (results.Count > 0)
  {
      OnItemsRealized(new ItemsRealizedEventArgs(startIndex, results.Count));
  }
}

Data coming back from the server is passed to UpdatePage. Before doing anything with the data, the method checks that it is still relevant – that the collection state hasn’t changed, and that the page to which the data belongs hasn’t been evicted from the cache. Then it takes each item and passes it to its corresponding VirtualItem.

Two snippets from VirtualItem will suffice to show what’s going on there:

public class VirtualItem<T> : INotifyPropertyChanged where T : class
{
    // Snip ...
    
   public T Item
   {
       get
       {
           if (!IsRealized && !DataFetchError)
           {
               _parent.RealizeItemRequested(Index);
           }
           return _item;
       }
       private set
       {
           _item = value;
           OnPropertyChanged(new PropertyChangedEventArgs("Item"));
           OnPropertyChanged(new PropertyChangedEventArgs("IsRealized"));
           IsStale = false;
       }
   }

   public void SupplyValue(T value)
   {
       DataFetchError = false;
       Item = value;
   }
}

VirtualItem makes the real item available through its Item property, and when SupplyValue is called this leads to a PropertyChanged event being raised for the Item property, so the Silverlight databinding infrastructure knows it needs to update the UI for that item.

Handling Updates

There’s one other important aspect of VirtualCollection to consider, and that is how it handles changes to the underlying collection of data.

It is the responsibility of the IVirtualCollectionSource to tell VirtualCollection when these changes happen.

The events are handled here:

private void HandleSourceCollectionChanged(object sender, VirtualCollectionSourceChangedEventArgs e)
{
  var stateWhenUpdateRequested = _state;
  if (e.ChangeType == ChangeType.Refresh)
  {
      Task.Factory.StartNew(() => UpdateData(stateWhenUpdateRequested), CancellationToken.None,
                            TaskCreationOptions.None, _synchronizationContextScheduler);
  }
  else if (e.ChangeType == ChangeType.Reset)
  {
      Task.Factory.StartNew(() => Reset(stateWhenUpdateRequested), CancellationToken.None,
                            TaskCreationOptions.None, _synchronizationContextScheduler);
  }
}

There are two types of changes we consider, a Refresh, and a complete Reset. Refresh happens when much of the data may well have remained unchanged, and it makes sense to continue to display it whilst we check with the server. A Reset happens when the entire collection has changed and displaying stale items would be inappropriate.

Notice that we are dispatching the events to be handled on the UI thread (there’s that _synchronizationContextScheduler again), and as we do so, we tag the request with the current collection state so that when we get round to handling the requests we can ignore any that are no longer relevant.

The Reset method contains nothing of great interest, but UpdateData explains something quite important:

protected void UpdateData(uint stateWhenUpdateRequested)
{
  if (_state != stateWhenUpdateRequested)
  {
      return;
  }

  _state++;

  MarkExistingItemsAsStale();

  _fetchedPages.Clear();
  _requestedPages.Clear();

  UpdateCount();

  var queryItemVisibilityArgs = new QueryItemVisibilityEventArgs();
  OnQueryItemVisibility(queryItemVisibilityArgs);

  if (queryItemVisibilityArgs.FirstVisibleIndex.HasValue)
  {
      var firstVisiblePage = queryItemVisibilityArgs.FirstVisibleIndex.Value / _pageSize;
      var lastVisiblePage = queryItemVisibilityArgs.LastVisibleIndex.Value / _pageSize;

      int numberOfVisiblePages = lastVisiblePage - firstVisiblePage + 1;
      EnsurePageCacheSize(numberOfVisiblePages);

      for (int i = firstVisiblePage; i <= lastVisiblePage; i++)
      {
          BeginGetPage(i);
      }
  }
  else
  {
      // in this case we have no way of knowing which items are currently visible,
      // so we signal a collection reset, and wait to see which pages are requested by the UI
      OnCollectionChanged(new NotifyCollectionChangedEventArgs(NotifyCollectionChangedAction.Reset));
  }
}

Remember that the purpose of UpdateData is to check that each item in the collection remains up to data. But it has to be frugal: it can’t go off to the server to ask about every possible item. Once again, it needs to limit itself to just those items that are visible.

One way of doing this, the way VirtualCollection worked when I first implemented it, is to raise a CollectionChanged event with an NotifyCollectionChangedAction of Reset. This signals to any listening ListBoxes or DataGrids that the whole collection has changed and they should rebuild their display of the data. They then respond by calling VirtualCollection’s indexer for each index  in the visible range, and thus VirtualCollection finds out which pages it should refresh. In many circumstances this is the only way of refreshing the collection because we have no other way of knowing exactly which items are visible.

But it is a rather sledgehammer approach, rebuilding all the rows in the UI, when we could just update the data within those rows, if only we knew which items are currently visible. So before resorting to a collection reset, VirtualCollection will try another approach. It tries asking nicely anybody who might be listening which items are visible.

It does this by raising the QueryItemVisibility event. To make sure this request doesn’t go unheeded, you need to attach a behaviour to your DataGrid or ListBox which will respond to the event. As part of the code sample, I have provided the ProvideVisibleItemRangeFromDataGridBehavior and the ProvideVisibleItemRangeFromItemsControlBehavior – though note that currently, this last behavior only does anything useful if the ItemsControl (usually in the form of a ListBox) is using a VirtualizingWrapPanel to display its items. See MainPage.xaml in the sample for an example of how to attach these behaviors.

The Final Word

And that concludes our dissection of VirtualCollection. Remember that you can try out a live demo here, and get all the code on github so that you can put it to use in your own projects.

Do let us know how you get on.

Next time: exploring the VirtualizingWrapPanel.

Data Virtualization, Lazy Loading, Stealth Paging–whatever you want to call it: Here’s how to do it in Silverlight

This is a guest post by Samuel Jack, a freelancer who has worked with us on the new RavenDb Studio features.

As I have previously mentioned, the management UI for RavenDb (fondly known as the Studio) has a hot new feature. A revolutionary interface for viewing huge sets of documents which uses (gasp!) scrollbars instead of paging controls. To make this feature kind on the server (we’re averse to killing Ravens of any kind), the UI requests documents just in time, as the users scrolls each part of the list into view.

In the next few blog posts, I want to share with you how we made that work in Silverlight. I’ve published a sample on GitHub, including code under a license that enables you to use it in your own projects.

Today, to whet your appetites, I’ll show you how easy it is to create user interfaces that display huge collections of items. In future posts, I’ll dig into the magic which makes it all work.

Flicking through the Netflix Catalogue

The sample is a simple browser for the Netflix movie catalogue, which lets you scroll through all the available titles, or just those containing the search terms you enter. I chose this example because Netflix have an OData API which makes the catalogue very easy to query.

As well as the details view, shown below, the sample also has a card view which demonstrates our VirtualizingWrapPanel, also included in the sample.

Netflix Browser Sample

Click the image to see the sample in action. (There’s a known issue with images not showing up, because of how I’m hosting the files in Amazon S3 – run the sample locally to see it in its full glory)

Holding it all together: VirtualCollection

The key player in this setup is the VirtualCollection class, which you’ll find in the sample code. VirtualCollection acts as the data source which you bind to your ListBoxes or DataGrids, and it has the responsibility of coordinating data retrieval from the server.  The actual fetching of data is handled by a class implementing IVirtualCollectionSource: each VirtualCollection is associated with one IVirtualCollectionSource.

public class MainViewModel : ViewModel
{
   private NetflixTitlesSource _source;
   public VirtualCollection<Title> Items { get; private set; }

   public MainViewModel()
   {
       _source = new NetflixTitlesSource();
       Items = new VirtualCollection<Title>(_source, pageSize: 20, cachedPages: 5);
   }

   protected override void OnViewLoaded()
   {
       Items.Refresh();
   }
}

Using the VirtualCollection is very straightforward. You create an instance, telling it what kind of items it will be managing (Title objects in this case), and supplying it with the IVirtualCollectionSource (implemented here by NetflixTitlesSource – more on this in a moment).

You also tell it what page size to use, in other words, how many items it should fetch in each call to the server. You can determine an appropriate page size by figuring out how many items are likely to fill one screen of your application. VirtualCollection also wants to know how many such pages it should cache in memory: if your items are small, and change infrequently, caching more pages might well be a good idea.

Supplying the goods: IVirtualCollectionSource

When it comes to implementing IVirtualCollectionSource, you have to do a little more work, but nothing too arduous. In fact, there’s a base class which does much of the housekeeping for you.

An IVirtualCollectionSource has two main responsibilities: it needs to inform the VirtualCollection of the total number of items, and it must supply pages of items when requested. It should also let the VirtualCollection know if anything changes, by means of the CollectionChanged event.

Here’s what the key methods of the NetflixTitlesSource look like:

public class NetflixTitlesSource : VirtualCollectionSource<Title>
{
    private string _search;

   public string Search
   {
       get { return _search; }
       set
       {
           _search = value;
           Refresh(RefreshMode.ClearStaleData);
       }
   }
   
   protected override Task<int> GetCount()
   {
       return GetQueryResults(0, 1, null)
           .ContinueWith(t => (int)t.Result.TotalCount, TaskContinuationOptions.ExecuteSynchronously);
   }

   protected override Task<IList<Title>> GetPageAsyncOverride(int start, int pageSize, IList<SortDescription> sortDescriptions)
   {
       return GetQueryResults(start, pageSize, sortDescriptions)
           .ContinueWith(t => (IList<Title>)((IEnumerable<Title>)t.Result).ToList(), TaskContinuationOptions.ExecuteSynchronously);
   }

    private Task<QueryOperationResponse<Title>> GetQueryResults(int start, int pageSize, IList<SortDescription> sortDescriptions)
   {
       var context = new NetflixCatalog(new Uri("http://odata.netflix.com/Catalog"));

       var orderByString = CreateOrderByString(sortDescriptions);
       var query = context.Titles
           .AddQueryOption("$skip", start)
           .AddQueryOption("$top", pageSize)
           .IncludeTotalCount();

       if (!string.IsNullOrEmpty(Search))
       {
           query = query.AddQueryOption("$filter", "(substringof('" + Search + "',Name) eq true) and (BoxArt/SmallUrl ne null)");
       }
       else
       {
           query = query.AddQueryOption("$filter", "(BoxArt/SmallUrl ne null)");
       }

       if (orderByString.Length > 0)
       {
           query = query.AddQueryOption("$orderby", orderByString);
       }

       return Task.Factory.FromAsync<IEnumerable<Title>>(query.BeginExecute, query.EndExecute, null)
           .ContinueWith(t => (QueryOperationResponse<Title>)t.Result, TaskContinuationOptions.ExecuteSynchronously);
   }
   
   ...
}

As you can see, the majority of the work work lies in preparing the query that we send off to the Netflix OData API, and executing that asynchronously . By comparison, the code needed to support IVirtualCollectionSource (just two methods, GetCount and GetPageAsyncOverride) is very small.

A VirtualCollectionSource can communicate with its parent VirtualCollection by means of the CollectionChanged event if something happens that means the collection needs to be updated. In this case, changing the Search property is going to change the entire collection, so we call the Refresh method on the base class, which raises that event.

There are two kinds of Refresh available. The Search property is using the ClearStaleData mode, which means that the whole collection will be cleared whilst new results are being loaded. This is obviously the right choice here, because continuing to show results for “Harry Potter” after the user has just changed the search terms to “Indiana Jones” would be confusing.

The other kind of Refresh is called PermitStateDataWhilstRefreshing. This mode is good if your server has just notified you that new items have been added. Existing items are still valid, so there’s no need to clear down the whole collection whilst you find out about the new items.

Making it Presentable

Finally, here’s some XAML to show how we might display the contents of the VirtualCollection in a DataGrid

<sdk:DataGrid ItemsSource="{Binding Items}" Grid.Row="1" Margin="10,0" IsReadOnly="True" AutoGenerateColumns="False" RowHeight="50">
 <i:Interaction.Behaviors>
     <VirtualCollection1:ProvideVisibleItemRangeFromDataGridBehavior/>
 </i:Interaction.Behaviors>
 <sdk:DataGrid.Columns>
     <sdk:DataGridTemplateColumn Header="Image">
         <sdk:DataGridTemplateColumn.CellTemplate>
             <DataTemplate>
                 <Image Stretch="None" 
                         Source="{Binding Item.BoxArt.SmallUrl, Converter={StaticResource Converter_StringToImage}}" 
                        Margin="5" />
             </DataTemplate>
         </sdk:DataGridTemplateColumn.CellTemplate>
     </sdk:DataGridTemplateColumn>
     <sdk:DataGridTextColumn Binding="{Binding Item.Name}" Header="Name" CanUserSort="True" SortMemberPath="Name"/>
     <sdk:DataGridTextColumn Binding="{Binding Item.AverageRating}" Header="Average Rating" CanUserSort="True" SortMemberPath="AverageRating"/>
     <sdk:DataGridTextColumn Binding="{Binding Item.ReleaseYear}" Header="Release Year" CanUserSort="True" SortMemberPath="ReleaseYear"/>
 </sdk:DataGrid.Columns>
</sdk:DataGrid>

The main thing to notice here is that all the binding paths start with Item. That’s because each data item in the VirtualCollection is wrapped in a VirtualItem, and made available through VirtualItem’s Item property. VirtualCollection does this so that it can put off fetching any actual items until the last possible moment – more on this in another blog post.

Notice also that we can enable sorting on some of the columns by setting a SortMemberPath. When you click a column header, the DataGrid will pass that value to the VirtualCollection, and VirtualCollection will pass it on to the IVirtualCollectionSource which can then use it to instruct the server in how to sort the results it returns.

Try it out and Get Back to Us

Now you’ve seen how easy it is, you have no excuse. Go and purge those paging controls from your apps! Long live scrollbars!

All the code you need is up on GitHub - look in the VirtualCollections folder. And don’t forget to check back here shortly, as we take a look behind the scenes to find out how VirtualCollection works.

We’d love to hear how you get on with this in your own projects.

New in RavenDb Studio 1.2: Low-Bandwidth mode

This is a guest post by Samuel Jack, a freelancer who has worked with us on the new RavenDb Studio features.

We saw earlier that in RavenDb Studio 1.2 you can now scroll through the entire list of documents in a Raven database, and the Studio will page in documents just in time behind the scenes. But those documents still need to be fetched from the server, and if your documents are very large (into the 100s of kilobytes and more), or you are working over a very slow connection, fetching the documents will take some time, and make for a frustrating experience.

Which is why we’ve introduced a low-bandwidth, or “Id only” mode for viewing document lists. You switch to it by clicking the Document View Style button on the top right of any Document list.

image

In this mode, the Studio will only fetch the metadata for each document, saving a lot of time when contacting the server and allowing the lists to be displayed much more quickly. To see the whole document, you simple double-click it, or click the Pencil icon, as in Details or Card mode.

Published at

Originally posted at

New in RavenDb Studio 1.2: Quick Links

This is a guest post by Samuel Jack, a freelancer who has worked with us on the new RavenDb Studio features.

Here’s another small, but perfectly formed feature in RavenDb Studio 1.2.

The header bar now sports a “quick links” area:

image

Two things to see here. First there’s the new New button which speeds you on your way to create a new document, index or query.

image

And there’s the “Go To Document” bar. Type the Id of a document, hit Enter, and you’ll be taken straight to it. Even better, if you don’t know the exact name of the document, start typing its Id, and you’ll be prompted with a list of possibilities.

image

Published at

Originally posted at

New in RavenDb Studio 1.2: Paging through result sets

This is a guest post by Samuel Jack, a freelancer who has worked with us on the new RavenDb Studio features.

A small, but very useful enhancement today, in our on-going tour through the new UI features for RavenDb 1.2

image

Notice the paging controls highlighted in the screenshot? These now appear whenever you arrive at a document via a documents list. This might be from the Documents or Collections page, for example, or from the results list of a Query. These controls let you page through all the documents in that particular list.

Say, for example, that you have misspelt an Artist’s name and you need to make a quick correction in the database. You could create a query to find all the documents containing the misspelling, and then double-click the results list to edit the first one. You make the correction, and save the document. What do you do next? Before, you would have had to hit the back button to go back to the query page to find the next document. Now you can just hit the Next button in the Edit page to work through the erroneous documents one-by-one.

New in RavenDb Studio 1.2: Better document editing

This is a guest post by Samuel Jack, a freelancer who has worked with us on the new RavenDb Studio features.

Continuing in our tour of the new features in RavenDb Studio for 1.2, today we come to the document editor.

Outlining Support

Since Documents are RavenDb’s raison d'etre, having a good document editing and viewing experience is vital. Which is why we’ve put a good bit of work into enhancing the Studio’s document editor.

The first thing you’ll notice is that the editor now has outlining support, so that you can expand and collapse collections and nested objects within a document:image_thumb35

Clicking the Outlining button collapses all the top level nodes, and clicking it again expands them all.

If you have many documents containing large collections, you may find the new Auto-Collapse Collections feature useful. When this mode is selected (find it by clicking the drop-down arrow on the Outlining button) the Document editor will automatically collapse all collections when you first open a document in the editor page.

JSON Syntax Error Highlighting

The next new feature you probably won’t notice, until you start modifying a document. Then you’ll see that the editor now has real-time JSON syntax checking:

image_thumb37

Any errors in the document are highlighted with red squiggles, and an error list at the bottom summaries all the errors detected, with double-click taking you to the mal-formed part of the document, as you’d expect.

Document Hyperlinks

Here’s a handy new feature for checking relationships between documents. The document editor now turns any document Ids it encounters within a document into hyperlinks that you can navigate to with a Shift-Click (or Shift-Ctrl-Click to open in a new page):

image

 

Not only is this good for jumping between related documents. It also lets you see at glance if your document references are valid, since the document editor only hyperlinks Ids of documents that actually exist in the database.

Published at

Originally posted at

New in RavenDb Studio 1.2: Enhancements to querying

This is a guest post by Samuel Jack, a freelancer who has worked with us on the new RavenDb Studio features.

Welcome back to our tour of the new features in RavenDb Studio 1.2. Up for examination today is the Query page.

Remembering Recent Queries

Over on the RavenDb issue tracker, we were asked to “treat queries as first-class citizens”. I don’t know what other rights queries might demand, but one right we decided to give them was the right to be remembered. Enter the Recent Queries dropdown:

image

Every query you execute is remembered, and you can go back to a previous query just by clicking its entry in the list. As with most navigation buttons in the Studio, clicking with the Control key held down will open the query in a new window.

Up to 100 of your most recent queries are remembered for next time after you close your browser. Though if you should have been querying with some nefarious purpose, there’s always the “Clear History” button to cover your tracks.

If you have any queries that are especially near and dear to your heart, you can Pin them to make sure they always stay at the top of the list and don’t get swept away if the rest of the history is cleared.

 

Other Changes

Along with this, there are a number of smaller enhancements

  • Sort By now works for dynamic queries when querying over a particular collection.
  • Similarly, you now get Intellisense in the query editor for dynamic queries when you have limited the query to a specific collection. In other words, press Ctrl-Space in the query editor, and you can choose from a list of properties in documents in that collection.
  • Sort By options are now remembered when you navigate back to a previous query.
  • You can now choose the default operator used to combine clauses in your query .
  • A Show Fields option has been added to help you understand what’s going on with your indexes. When pressed, the results view will show the fields in matching index entries, rather than the actual documents. In a similar vein, there’s a new Skip Transform button which shows up when an Index has a Transform stage defined. Pressing Skip Transform will show you the documents matching a query, rather than the results of the transform.
  • For all you JSON-heads out there, the Query page shows you what url the client is calling to get the results from the server, and if you press the link button next to the url, you’ll see the raw JSON that the server sends back.

New in RavenDb Studio 1.2: Scrollable Document Lists, and a Details View

This is a guest post by Samuel Jack, a freelancer who has worked with us on the new RavenDb Studio features.

Along with the exciting new features in the core RavenDb database, we’ve been hard at work improving RavenDb Studio, the management UI for RavenDb. Over the next few days, I want to share with you some of the major enhancements.

Death to paging!

Almost the very first thing we did when we started work on the Studio for 1.2 was to kill the paging interface for document lists. And hammer nails in its coffin. Please give a big round of applause as we introduce … scrollbars!

image_thumb3

That’s right: scrollbars that let you scroll through your entire collection of documents, all 2,147,483,647 of them. Don’t be alarmed, you db admin types: the Studio is doing the right thing and paging in documents only as you scroll to them (and paging out documents you’ve scrolled past, so they don’t clog up memory).

 

Digging into the Document Details

By popular request, we’ve also added a details view as an alternative to the existing card view. Click the Document View Style button in the top right corner of any documents list and you can toggle between card view and details view (which ever setting you choose is remembered between sessions).

image_thumb12

Which columns get shown in details view? Well, naturally the Studio allows you to choose for yourself which columns you want to see. Or you can let the Studio choose for you.

 

 

Pick your own Columns

To choose your own columns, right click on one of the column headers and click the Choose Columns menu item. You’ll see a dialog where you can choose the columns you’d like:

image_thumb17

When you click in the Binding column to add a new Column, the Studio will help you out by showing a list of all the properties (and nested properties) in the visible documents:

image_thumb19

Notice that you can bind to document meta-data too!

 

Automatic Column Selection

To save you having to pick columns yourself for each set of documents you view, the Studio has a special IntelliPickNot TM feature. It will inspect the properties of documents currently scrolled into view, and do its best to pick a sensible set of columns. And as you scroll, and different types of documents come into view, it will automatically update the column set.

How does it choose columns? Good question. Let me check the documentation source code…

Basically it picks the top 6 most commonly occurring properties in the current set of documents. But it will give certain properties a boost in importance making them more likely to be picked. The Studio has a built in list of property names that it will look for. So any property that includes the words “Name”, “Title” “Description”, or “Status” will be prioritised. You can customise this list of priority columns for a database  by creating a document  in that database called “Raven/Studio/PriorityColumns”, like this:

image_thumb22

PropertyNamePattern is interpreted as a Regular Expression so you can get quite sophisticated in properties you want to boost.

The IntelliPick algorithm has one more trick up its sleeve. Have a look at this screenshot:

image_thumb30

Do you see it? If you include particular properties in a query, or as a Sort By option, those properties will get boosted in importance. In the example shown, Artist.Name is part of the query, and results are sorted by Genre.Name, so both these properties are picked out to appear as columns. Similarly, when viewing results for a particular index, any properties that are included in the Index name will also be given a boost in priority.

Stress testing RavenDB

The following is cross posted from Mark Rodseth’s blog (he also posted a follow up post with more details).

Mark is a .Net Technical Architect at a digital agency named Fortune Cookie in London. I would like to take the opportunity and thank Mark both for the grand experiment about which you are about to read and for the permission to post this in the company blog.

Please note: Mark or Fortune Cookie are not affiliated with either Hibernating Rhinos or RavenDB in any way.

When a colleague mentioned  RavenDB  to me I had a poke around and discovered that it was one of the more popular open source NoSQL technologies on the market. Not only that but it was bundled with Lucene.Net Search making it Document Database coupled with Lucene search capabilities.  With an interest in NoSQL technology and a grudge match that hadn’t been settled with Lucene.Net, I set myself the challenge to swap out our SQL Search implementation with RavenDB and then do a like for like load test against the two search technologies.
These are my findings from both a programmatic and performance perspective.


Installing RavenDB
There isnt much to installing Raven and its pretty much a case of downloading the latest build and running the Server application.
The server comes with a nice Silverlight management interface which allows you to manage all aspects of Raven Db from databases to data to indexes. All tasks have a programmatic equivalent but a decent GUI is an essential tool for noobs like myself.

Storing the Data
My first development task was to write an import routine which parsed the property data in SQL and then add it into a Raven Database. This was fairly easy and all I needed to do was to create a POCO, plug it with data from SQL and save it using the C# Raven API. The POCO serialised into JSON data and saved as a new document in the  RavenDB.

The main challenge here was changing my thinking from relational modelling to domain driven modelling - a paradigm shift required when moving to NoSQL - which includes concepts like aggregate roots, entities and value types. Journeying into this did get a bit metaphysical at times but here is my understanding of this new fangled schism.

Entity - An entity is something that has a unique identity and meaning in both the business and system context. In the property web site example, a flat or a bungalow or an office match these criteria.

Value Type - Part of the entity which does not require its own identity and has no domain or system relevance on its own. For example, a bedroom or a toilet.

Aggregate Root - Is an master entity with special rules and access permissions that relate to a grouping of similar entities. For example, a property is an aggregate of flats, bungalows and offices. This is the best description of these terms I found.

Hibernating Rhinos note: With RavenDB, we usually consider the Entity and Aggregate Root to be synonyms to a Document. There isn’t a distinction in RavenDB between the two, and they map to a RavenDB document.

In this example, I created one Aggregate Root Entity to store all property types.

C# Property POCO

Indexing the Data
Once the Data was stored it needed to be indexed for fast search. To achieve this I had to get to grips with map reduce functions which I had seen around but avoided like the sad and lonely looking bloke** at a FUN party.
The documentation is pretty spartan on the  RavenDB web site but after hacking away I finally created an index that worked on documents with nested types and allowed for spatial queries.
RavenDB allows you to create indexes using Map Reduce functions in LINQ. What this allows you to do is create a Lucene index from a large, tree like structure of data. Map reduce functions give you the same capability as SQL using joins and group by statements. To create a spatial index which allowed me to search properties by type and sector (nested value types) I created an index using the following Map Reduce function.

Index created using the Raven DB Admin GUI

Hibernating Rhinos note: a more efficient index would likely be something like:

from r in docs.AssetDetailPocos
select new
{
  sectorname = r.Sectors,
  prnlname = r.AddressPnrls,

  r.AssetId,
  r.AskingPrice,
  r.NumberOfBedrooms,
  r.NumberOfBathRooms,
  
  
  _ = SpatialIndex.Generate(r.AssetLatitude, r.AssetLongitude)
}

This would reduce the number of index entries and make the index smaller and faster to generate.

Querying the data

Now that I had data that was indexed, the final development challenge was querying it. RavenDB has a basic search API and a Lucene Query API for more complex queries. Both allow you to write queries in LINQ. To create the kind if complex queries you would require in a property searching web site, the API was a bit lacking. To work around this I had to construct my own native Lucene queries. Fortunately the API allowed me to do so.

Performance Testing

All the pawns were now in place for my load test.

  • The entire property SQL database was mirrored to  RavenDB.
  • The Search Interface now had both a SQL and a  RavenDB implementation.
  • I created a crude Web Page which allowed switching the search from SQL to  RavenDB via query string parameters and output the results using paging.To ensure maximum thrashing the load tests passed in random geo locations for proximity search and keywords for attribute search. 
  • A VM was setup and ready to face the wrath of BrowserMob.

I created a ramp test scaling from 0 to 1000 concurrent users firing a single get request with no think time at the Web Page and ran it in isolation against the SQL Implementation and then in isolation against the  RavenDB Implementation. The test ran for 30 minutes.
And for those of you on the edge of you seat the results where a resounding victory for  RavenDB. Some details of the load test are below but the headline is SQL choked at 250 concurrent users whereas with  RavenDB even with 1000 concurrent users the response time was below 12 seconds.

SQL Load Test

Transactions: 111,014 (Transaction = Single Get Request)
Failures: 110,286 (Any 500 or timeout)

SQL Data Throughput - Flatlines at around 250 concurrent users.

RavenDB Load Test

Transactions: 145,554 (Transaction = Single Get Request)
Failures: 0 (Any 500 or timeout)

RavenDB Data Throughput - What the graph should look like

Final thoughts

RavenDB is a great document database with fairly powerful search capabilities. It has a lot of pluses and a few negative which are listed for you viewing pleasure below.
Positives

  • The documentation although spartan does cover the fundamentals making it easy to get started. On some instances I did have to sniff through the source code to fathom how some things worked but that is the beauty of open source I guess. 
  • The Silverlight Admin interface is pretty sweet 
  • The Raven community (a google group) is very active and the couple of queries I posted were responded to almost immediately.
  • Although the API did present some challenges it both allowed you to bypass its limitations and even contribute yourself to the project.
  • The commercial licence for  RavenDB is pretty cheap at a $600 once off payment

Negatives

  • The web site documentation and content could do with an a facelift. (Saying that, I just checked the web site and it seems to have been be revamped)
  • I came a cross a bug in the Lucene.Net related to granular spatial queries which has yet to be resolved.   Not  RavenDB's fault but a dependence on third party libraries can cause issues. 
  • I struggled to find really impressive commercial reference sites. There are some testimonials but they give little information away. 
  • Sharding scares me.

I look forward to following the progress of  RavenDB and hopefully one day using it in a commercial project. I'm not at the comfort level yet for proposing it but with some more investigation and perhaps some good reference sites this could change very quickly.


* Starry Eyed groupies sadly didn't exist, nor have they ever.
** Not me.

http://ravendb.net

Tags:

Published at

Originally posted at

Invalid JSON results via Powershell 3

This is a guest post by Chris, a RavenDB user, originally posted on the RavenDB group.

Background

I’ve been fiddling with document databases for a while trying to find a way to aggregate our logs without having to demoralize everything. I started with CouchDB and because of the odd status of the project with the founder abandoning it and since I work in a Windows world, going with Raven ultimately seemed like a wholly better solution – all the goods of Couch plus a .Net API.

I built a data model in C# and used a CSV library to push a few hundred thousand records into Raven. The goal is to allow support engineers to query this log information with some simple PowerShell one-liners.

I thought parsing 100megs of CSV and jamming in the thousands of corresponding documents would be the difficult part to figure out and the one-liners would be a couple hours of work at most.

The Problem

Yes, it was a bit challenging to build an efficient importer, but most of that was due to my limited knowledge of C#. I thought it was still not working when I started to build my Posh one-liners and some queries were returning invalid JSON. The PowerShell 3 CTP introduce some new cmdlets for handling JSON and one specifically for executing HTTP Rest API calls – Invoke-RestMethod (alias irm). IRM is 100 times more user friendly than CURL and it even goes a step above and beyond by automatically deserializing the content returned. So, it’s extremely easy to consume JSON via a REST API, but I kept running into an issue where Posh would throw an error that there was no Results (the deserialized JSON) property on the object. I must have been pushing in bad data – garbage in equals garbage out.

I thought the HTML in some of the log values was invalidating the JSON string. I deleted all the documents, added a call to HTTPEncode on all strings before Storing it to Raven, then I reloaded the records. I ended up with the same results; or, lack of Results property in this case.

The Invoke-RestMethod cmdlet did return what looked like a valid JSON string, so I dropped it into Notepad++ and turned on JavaScript syntax highlighting and went through the string. I found the culprit.

"SkippedResults:0,

There was a missing quote around a key name. I almost posted this as a RavenDB bug, but I checked the raw HTTP response in Fiddler and found that the quote existed.

What do to about it

I am working with a Technology Preview release of Powershell 3, so I checked Connect and found a similar report of missing characters in content deserialization.

“It seems that the last byte of the second HTTP response packet is being dropped by powershell. This only seems to happen when the second response packet is smaller than the first.”

Microsoft closed this issue as “Won’t Fix” and suggested using Invoke-WebRequest instead. Invoke-WebRequest removes all the magic of Invoke-RestMethod and just gets raw HTTP content. I piped the Content parameter to the, also new to Powershell 3, cmdlet ConvertFrom-Json and got proper deserializedJSON.

$c = Invoke-WebRequest "http://localhost:8080/indexes/vovici/Logs/process?query=customer:blah"
$r = $c.content | convertfrom-json

So, it’s an extra step, but if you want to reliably access RavenDB’s HTTP API via PowerShell 3 that seems to be the way to do it.

UPDATE: It does look like MS fixed the lost character issue in the Invoke-RestMethod cmdlet in the latest beta release. I can pull down 1024 documents with one web request and get a usable, deserialized Object[].

Published at

Originally posted at

XAML Magic: Turning an Ugly Silverlight Duckling into a Beautiful Photoshopped Swan

This is a guest post by Samuel Jack, who had done a lot of work on the new UI for RavenDB.

Three weeks ago Ayende put out a request for help in turning an ugly duckling into a beautiful swan, and I, rather nervously, signed up for the job. The ugly duckling in question was Raven Studio, the Silverlight-based management UI for Raven Db. The nerves were a result of doubting that my limited graphic design skills were up to the job. But when Ayende assured me that he had a proto-type swan in the form of a Photoshop design, drawn up by a bona-fide, turtle-neck-wearing designer, they were calmed. Marginally.

Because the design Ayende had was for the new-look RavenDb website.

image

He wanted me to take the look and feel and transfer it to the Silverlight Raven Studio application. Which, when he handed it over to me, looked like this:

image

Ahh! Where to start?

Photoshop for Developer Dummies

To ease myself in, I got started by simply trying to imitate parts of the Photoshop design in XAML, beginning with the header bar at the very top of the page. Not being a designer myself, I’m rather like a duck out of water when it comes to Photoshop, but I’ve at least got the basics sussed.

The thing to understand is that designers construct Photoshop images like onions, layer upon layer, sometimes eye-watering in complexity, and to reproduce the design, you have to peel down through the layers.

Photoshop Layer BasicsFirst go to Photoshop’s Layers pane, and make sure all the layers are unlocked. This allows the Move Layer tool to come into play, not to move layers, but identify layers by selecting them in the Layers pane when you click the corresponding part of the image. Once you’ve identified a layer, Alt-Click it, and all other layers in the image will be hidden, allowing you to figure out exactly how the thing should look.

Mostly when I’m paring down Photoshop layers I’m looking to isolate them so that I can figure out the colour gradients they use. You could, of course, navigate your way through Photoshop’s dialogs to read off the exact RGB values. Or, if you can get the layer on its lonesome, you can use Expression Blend’s Gradient Dropper tool. GradientEyeDropper

This is a brilliant little timesaver. In the Blend Property pane, select the Brush property of your choice, put it into Gradient mode, click the Gradient Dropper tool, then drag over any area on screen, and Blend will reproduce the gradient under the cursor in XAML form.

After all that, I have the first feather of the new swan: a header bar matching the Photoshop design. Well, the background of the header bar. It needs fleshing out with some buttons.

Let it Live: Control Templates and Visual States

Silverlight, following WPF, has the concept of look-less controls. That is, the Controls (take Button as an example) manage their behaviour (Buttons respond to mouse clicks by executing commands) but don’t define how they are rendered on screen. That is left to the control’s Style, and specifically its ControlTemplate. The ControlTemplate defines the visual tree of all the UI elements needed to present the control and make it look snazzy. With a little patience, some assistance from Expression Blend, and plenty of application of the Gradient Dropper tool, it’s possible to take the built-in controls and make them look and feel just how the designer ordered.

I wanted Buttons that look like those in the header bar of the Photoshop design, but when the corresponding page is selected, they should change to have a background gradient with colours like the RavenDb logo.

When restyling a Control, it’s best to start by modifying the existing style. This way you can be sure you won’t miss an aspect of the control’s behaviour that you might otherwise forget. Blend makes this easy by giving you the option of editing a copy of the Control’s current ControlTemplate (right-click on it in the Objects and Timeline View, then select Edit Template > Edit a Copy). There are occasions when that little trick has failed, and I’ve ended up with an empty control template. But MSDN has come through for me then: it has a whole section containing the default Styles and Control Templates for all the built-in controls, like this one for Button.

Part of the ControlTemplate defines how the control looks in its various states, when the mouse is over it, when it has focus, or when it is selected, for example. The Control itself is responsible for indicating when it has entered each state. As a designer, it’s your job to specify Storyboards that are activated each time particular states are entered. Each Storyboard can animate whichever properties it likes to achieve the desired effect – in my Buttons, for example, I animate the opacity property of a Border element to fade in a coloured background indicating that it is selected. All this is overseen by the VisualStateManager, of which, more here. Naturally, Expression Blend has great editing support for visual states. Read John Papa’s tutorial to learn more.

So now I have a header bar with buttons that change colour when the corresponding page is selected. Where next?

Textured Backgrounds

Well, that page background could do with spicing up. The Photoshop design has a nice textured background, which I extracted to a PNG file that Silverlight would understand by hiding every layer except the background, then using Photoshop’s “Save for Web & Devices” feature.

The thing about textured backgrounds is that you do want them to cover the whole of the page background, which means tiling the texture to fill all the space. WPF makes this easy with its ImageBrush, which has a TileMode property, which, when set to a value other than None, automatically repeats the image over the whole area to be painted by the brush. Silverlight has ImageBrushes, but they don’t support tiling out of the box. Fortunately, Phil Middlemiss has supplied what is lacking in the form of the TiledBGControl which does exactly what I need – you should take a look: it makes clever use of Silverlight’s pixel shader effects.

This is what we’ve got so far.

image

The Index Editor Page, Before and After

Here are a couple of other pages I’ve beautified. First, the Index Editor page, as it was before:

image

And now:

image

Again, it was a challenge knowing where to apply my beautician’s brush first. I settled on adding the header bar at the top, and I then realised it could double up as a toolbar. Originally the page had no header at all, but by having a bread-crumb bar in the header it helps to give the user a bit more context when they’re looking at the page, as well as making it easier to navigate around.

Inspiration and Icons

Since my graphic-design skills are so underdeveloped, I borrow ideas shamelessly wherever I find something that fits. You may recognise the styling of the toolbar buttons in the Index page header bar as being remarkably similar to the ones on the Google Chrome toolbar. Yes – Expression Blend’s Gradient dropper does work on live applications too! Two places to check out if you find yourself short on inspiration are Quince and Pattern Tab which both catalogue examples of user interface and user experience design from across the web. Pattern Tab especially has myriad examples of beautiful UIs.

In the past I’ve struggled to find icons for my projects, but I’ve recently discovered two great sources: IconFinder.com and IconArchive.com. Both have excellent search facilities (which is often what’s missing from the commercial collections you buy and download to your machine in a whacking great zip file), and are careful to call out the license attached to each icon. A surprisingly large number are licensed so that they can be used without charge in commercial products.

A XAML Tip

The nice thing about styling an application is that it gets easier with every page you complete. Once you’ve settled on a look for a particular kind of element, you can repeat that look on every page. Silverlight’s support for Styles and Resources makes this very easy. And I have a tip that can make it easier still.

I put all my styles into a single resource dictionary, Styles.xaml which I merge into my App.xaml resource dictionary. I then name all my Styles, Brushes, etc. using a hierarchical naming convention. So Styles all begin with “Style_”, Styles for Buttons would all begin “Style_Button”, and then would come the styles for different purposes: “Style_Button_Toolbar”, “Style_Button_Hero” (for those big red buttons in your app that the hero uses to save the world), etc.. The pay-off for using this convention comes when you’re hand-editing XAML and making use of Resharper’s XAML intellisense. Type “{StaticResource Style_[ControlType]” and Resharper instantly presents you with a list of all the styles that might apply.

A Parting Screenshot

To finish, here’s one more before and after comparison, this time of the Edit Document page. Before:

Edit Document Page - Before

And after:

image

You can begin to sense the benefit of using a consistent set of styles, as it brings a harmonious feel to the whole application.

I hope you’ve enjoyed this whistle-stop tour of the Raven Studio beautification process. Remember that all the code is available on GitHub. We’d love to hear what you think.

Tags:

Published at

Originally posted at

XAML Magic: Turning an Ugly Silverlight Duckling into a Beautiful Photoshopped Swan

This is a guest post by Samuel Jack, who had done a lot of work on the new UI for RavenDB.

Three weeks ago Ayende put out a request for help in turning an ugly duckling into a beautiful swan, and I, rather nervously, signed up for the job. The ugly duckling in question was Raven Studio, the Silverlight-based management UI for Raven Db. The nerves were a result of doubting that my limited graphic design skills were up to the job. But when Ayende assured me that he had a proto-type swan in the form of a Photoshop design, drawn up by a bona-fide, turtle-neck-wearing designer, they were calmed. Marginally.

Because the design Ayende had was for the new-look RavenDb website.

image

He wanted me to take the look and feel and transfer it to the Silverlight Raven Studio application. Which, when he handed it over to me, looked like this:

image

Ahh! Where to start?

Photoshop for Developer Dummies

To ease myself in, I got started by simply trying to imitate parts of the Photoshop design in XAML, beginning with the header bar at the very top of the page. Not being a designer myself, I’m rather like a duck out of water when it comes to Photoshop, but I’ve at least got the basics sussed.

The thing to understand is that designers construct Photoshop images like onions, layer upon layer, sometimes eye-watering in complexity, and to reproduce the design, you have to peel down through the layers.

Photoshop Layer BasicsFirst go to Photoshop’s Layers pane, and make sure all the layers are unlocked. This allows the Move Layer tool to come into play, not to move layers, but identify layers by selecting them in the Layers pane when you click the corresponding part of the image. Once you’ve identified a layer, Alt-Click it, and all other layers in the image will be hidden, allowing you to figure out exactly how the thing should look.

Mostly when I’m paring down Photoshop layers I’m looking to isolate them so that I can figure out the colour gradients they use. You could, of course, navigate your way through Photoshop’s dialogs to read off the exact RGB values. Or, if you can get the layer on its lonesome, you can use Expression Blend’s Gradient Dropper tool. GradientEyeDropper

This is a brilliant little timesaver. In the Blend Property pane, select the Brush property of your choice, put it into Gradient mode, click the Gradient Dropper tool, then drag over any area on screen, and Blend will reproduce the gradient under the cursor in XAML form.

After all that, I have the first feather of the new swan: a header bar matching the Photoshop design. Well, the background of the header bar. It needs fleshing out with some buttons.

Let it Live: Control Templates and Visual States

Silverlight, following WPF, has the concept of look-less controls. That is, the Controls (take Button as an example) manage their behaviour (Buttons respond to mouse clicks by executing commands) but don’t define how they are rendered on screen. That is left to the control’s Style, and specifically its ControlTemplate. The ControlTemplate defines the visual tree of all the UI elements needed to present the control and make it look snazzy. With a little patience, some assistance from Expression Blend, and plenty of application of the Gradient Dropper tool, it’s possible to take the built-in controls and make them look and feel just how the designer ordered.

I wanted Buttons that look like those in the header bar of the Photoshop design, but when the corresponding page is selected, they should change to have a background gradient with colours like the RavenDb logo.

When restyling a Control, it’s best to start by modifying the existing style. This way you can be sure you won’t miss an aspect of the control’s behaviour that you might otherwise forget. Blend makes this easy by giving you the option of editing a copy of the Control’s current ControlTemplate (right-click on it in the Objects and Timeline View, then select Edit Template > Edit a Copy). There are occasions when that little trick has failed, and I’ve ended up with an empty control template. But MSDN has come through for me then: it has a whole section containing the default Styles and Control Templates for all the built-in controls, like this one for Button.

Part of the ControlTemplate defines how the control looks in its various states, when the mouse is over it, when it has focus, or when it is selected, for example. The Control itself is responsible for indicating when it has entered each state. As a designer, it’s your job to specify Storyboards that are activated each time particular states are entered. Each Storyboard can animate whichever properties it likes to achieve the desired effect – in my Buttons, for example, I animate the opacity property of a Border element to fade in a coloured background indicating that it is selected. All this is overseen by the VisualStateManager, of which, more here. Naturally, Expression Blend has great editing support for visual states. Read John Papa’s tutorial to learn more.

So now I have a header bar with buttons that change colour when the corresponding page is selected. Where next?

Textured Backgrounds

Well, that page background could do with spicing up. The Photoshop design has a nice textured background, which I extracted to a PNG file that Silverlight would understand by hiding every layer except the background, then using Photoshop’s “Save for Web & Devices” feature.

The thing about textured backgrounds is that you do want them to cover the whole of the page background, which means tiling the texture to fill all the space. WPF makes this easy with its ImageBrush, which has a TileMode property, which, when set to a value other than None, automatically repeats the image over the whole area to be painted by the brush. Silverlight has ImageBrushes, but they don’t support tiling out of the box. Fortunately, Phil Middlemiss has supplied what is lacking in the form of the TiledBGControl which does exactly what I need – you should take a look: it makes clever use of Silverlight’s pixel shader effects.

This is what we’ve got so far.

image

The Index Editor Page, Before and After

Here are a couple of other pages I’ve beautified. First, the Index Editor page, as it was before:

image

And now:

image

Again, it was a challenge knowing where to apply my beautician’s brush first. I settled on adding the header bar at the top, and I then realised it could double up as a toolbar. Originally the page had no header at all, but by having a bread-crumb bar in the header it helps to give the user a bit more context when they’re looking at the page, as well as making it easier to navigate around.

Inspiration and Icons

Since my graphic-design skills are so underdeveloped, I borrow ideas shamelessly wherever I find something that fits. You may recognise the styling of the toolbar buttons in the Index page header bar as being remarkably similar to the ones on the Google Chrome toolbar. Yes – Expression Blend’s Gradient dropper does work on live applications too! Two places to check out if you find yourself short on inspiration are Quince and Pattern Tab which both catalogue examples of user interface and user experience design from across the web. Pattern Tab especially has myriad examples of beautiful UIs.

In the past I’ve struggled to find icons for my projects, but I’ve recently discovered two great sources: IconFinder.com and IconArchive.com. Both have excellent search facilities (which is often what’s missing from the commercial collections you buy and download to your machine in a whacking great zip file), and are careful to call out the license attached to each icon. A surprisingly large number are licensed so that they can be used without charge in commercial products.

A XAML Tip

The nice thing about styling an application is that it gets easier with every page you complete. Once you’ve settled on a look for a particular kind of element, you can repeat that look on every page. Silverlight’s support for Styles and Resources makes this very easy. And I have a tip that can make it easier still.

I put all my styles into a single resource dictionary, Styles.xaml which I merge into my App.xaml resource dictionary. I then name all my Styles, Brushes, etc. using a hierarchical naming convention. So Styles all begin with “Style_”, Styles for Buttons would all begin “Style_Button”, and then would come the styles for different purposes: “Style_Button_Toolbar”, “Style_Button_Hero” (for those big red buttons in your app that the hero uses to save the world), etc.. The pay-off for using this convention comes when you’re hand-editing XAML and making use of Resharper’s XAML intellisense. Type “{StaticResource Style_[ControlType]” and Resharper instantly presents you with a list of all the styles that might apply.

A Parting Screenshot

To finish, here’s one more before and after comparison, this time of the Edit Document page. Before:

Edit Document Page - Before

And after:

image

You can begin to sense the benefit of using a consistent set of styles, as it brings a harmonious feel to the whole application.

I hope you’ve enjoyed this whistle-stop tour of the Raven Studio beautification process. Remember that all the code is available on GitHub. We’d love to hear what you think.

Tags:

Published at

Originally posted at

Comments (1)

XAML Magic: Turning an Ugly Silverlight Duckling into a Beautiful Photoshopped Swan

This is a guest post by Samuel Jack, who had done a lot of work on the new UI for RavenDB.

Three weeks ago Ayende put out a request for help in turning an ugly duckling into a beautiful swan, and I, rather nervously, signed up for the job. The ugly duckling in question was Raven Studio, the Silverlight-based management UI for Raven Db. The nerves were a result of doubting that my limited graphic design skills were up to the job. But when Ayende assured me that he had a proto-type swan in the form of a Photoshop design, drawn up by a bona-fide, turtle-neck-wearing designer, they were calmed. Marginally.

Because the design Ayende had was for the new-look RavenDb website.

image

He wanted me to take the look and feel and transfer it to the Silverlight Raven Studio application. Which, when he handed it over to me, looked like this:

image

Ahh! Where to start?

Photoshop for Developer Dummies

To ease myself in, I got started by simply trying to imitate parts of the Photoshop design in XAML, beginning with the header bar at the very top of the page. Not being a designer myself, I’m rather like a duck out of water when it comes to Photoshop, but I’ve at least got the basics sussed.

The thing to understand is that designers construct Photoshop images like onions, layer upon layer, sometimes eye-watering in complexity, and to reproduce the design, you have to peel down through the layers.

Photoshop Layer BasicsFirst go to Photoshop’s Layers pane, and make sure all the layers are unlocked. This allows the Move Layer tool to come into play, not to move layers, but identify layers by selecting them in the Layers pane when you click the corresponding part of the image. Once you’ve identified a layer, Alt-Click it, and all other layers in the image will be hidden, allowing you to figure out exactly how the thing should look.

Mostly when I’m paring down Photoshop layers I’m looking to isolate them so that I can figure out the colour gradients they use. You could, of course, navigate your way through Photoshop’s dialogs to read off the exact RGB values. Or, if you can get the layer on its lonesome, you can use Expression Blend’s Gradient Dropper tool. GradientEyeDropper

This is a brilliant little timesaver. In the Blend Property pane, select the Brush property of your choice, put it into Gradient mode, click the Gradient Dropper tool, then drag over any area on screen, and Blend will reproduce the gradient under the cursor in XAML form.

After all that, I have the first feather of the new swan: a header bar matching the Photoshop design. Well, the background of the header bar. It needs fleshing out with some buttons.

Let it Live: Control Templates and Visual States

Silverlight, following WPF, has the concept of look-less controls. That is, the Controls (take Button as an example) manage their behaviour (Buttons respond to mouse clicks by executing commands) but don’t define how they are rendered on screen. That is left to the control’s Style, and specifically its ControlTemplate. The ControlTemplate defines the visual tree of all the UI elements needed to present the control and make it look snazzy. With a little patience, some assistance from Expression Blend, and plenty of application of the Gradient Dropper tool, it’s possible to take the built-in controls and make them look and feel just how the designer ordered.

I wanted Buttons that look like those in the header bar of the Photoshop design, but when the corresponding page is selected, they should change to have a background gradient with colours like the RavenDb logo.

When restyling a Control, it’s best to start by modifying the existing style. This way you can be sure you won’t miss an aspect of the control’s behaviour that you might otherwise forget. Blend makes this easy by giving you the option of editing a copy of the Control’s current ControlTemplate (right-click on it in the Objects and Timeline View, then select Edit Template > Edit a Copy). There are occasions when that little trick has failed, and I’ve ended up with an empty control template. But MSDN has come through for me then: it has a whole section containing the default Styles and Control Templates for all the built-in controls, like this one for Button.

Part of the ControlTemplate defines how the control looks in its various states, when the mouse is over it, when it has focus, or when it is selected, for example. The Control itself is responsible for indicating when it has entered each state. As a designer, it’s your job to specify Storyboards that are activated each time particular states are entered. Each Storyboard can animate whichever properties it likes to achieve the desired effect – in my Buttons, for example, I animate the opacity property of a Border element to fade in a coloured background indicating that it is selected. All this is overseen by the VisualStateManager, of which, more here. Naturally, Expression Blend has great editing support for visual states. Read John Papa’s tutorial to learn more.

So now I have a header bar with buttons that change colour when the corresponding page is selected. Where next?

Textured Backgrounds

Well, that page background could do with spicing up. The Photoshop design has a nice textured background, which I extracted to a PNG file that Silverlight would understand by hiding every layer except the background, then using Photoshop’s “Save for Web & Devices” feature.

The thing about textured backgrounds is that you do want them to cover the whole of the page background, which means tiling the texture to fill all the space. WPF makes this easy with its ImageBrush, which has a TileMode property, which, when set to a value other than None, automatically repeats the image over the whole area to be painted by the brush. Silverlight has ImageBrushes, but they don’t support tiling out of the box. Fortunately, Phil Middlemiss has supplied what is lacking in the form of the TiledBGControl which does exactly what I need – you should take a look: it makes clever use of Silverlight’s pixel shader effects.

This is what we’ve got so far.

image

The Index Editor Page, Before and After

Here are a couple of other pages I’ve beautified. First, the Index Editor page, as it was before:

image

And now:

image

Again, it was a challenge knowing where to apply my beautician’s brush first. I settled on adding the header bar at the top, and I then realised it could double up as a toolbar. Originally the page had no header at all, but by having a bread-crumb bar in the header it helps to give the user a bit more context when they’re looking at the page, as well as making it easier to navigate around.

Inspiration and Icons

Since my graphic-design skills are so underdeveloped, I borrow ideas shamelessly wherever I find something that fits. You may recognise the styling of the toolbar buttons in the Index page header bar as being remarkably similar to the ones on the Google Chrome toolbar. Yes – Expression Blend’s Gradient dropper does work on live applications too! Two places to check out if you find yourself short on inspiration are Quince and Pattern Tab which both catalogue examples of user interface and user experience design from across the web. Pattern Tab especially has myriad examples of beautiful UIs.

In the past I’ve struggled to find icons for my projects, but I’ve recently discovered two great sources: IconFinder.com and IconArchive.com. Both have excellent search facilities (which is often what’s missing from the commercial collections you buy and download to your machine in a whacking great zip file), and are careful to call out the license attached to each icon. A surprisingly large number are licensed so that they can be used without charge in commercial products.

A XAML Tip

The nice thing about styling an application is that it gets easier with every page you complete. Once you’ve settled on a look for a particular kind of element, you can repeat that look on every page. Silverlight’s support for Styles and Resources makes this very easy. And I have a tip that can make it easier still.

I put all my styles into a single resource dictionary, Styles.xaml which I merge into my App.xaml resource dictionary. I then name all my Styles, Brushes, etc. using a hierarchical naming convention. So Styles all begin with “Style_”, Styles for Buttons would all begin “Style_Button”, and then would come the styles for different purposes: “Style_Button_Toolbar”, “Style_Button_Hero” (for those big red buttons in your app that the hero uses to save the world), etc.. The pay-off for using this convention comes when you’re hand-editing XAML and making use of Resharper’s XAML intellisense. Type “{StaticResource Style_[ControlType]” and Resharper instantly presents you with a list of all the styles that might apply.

A Parting Screenshot

To finish, here’s one more before and after comparison, this time of the Edit Document page. Before:

Edit Document Page - Before

And after:

image

You can begin to sense the benefit of using a consistent set of styles, as it brings a harmonious feel to the whole application.

I hope you’ve enjoyed this whistle-stop tour of the Raven Studio beautification process. Remember that all the code is available on GitHub. We’d love to hear what you think.

Tags:

Published at

Originally posted at

Comments (2)

XAML Magic: Turning an Ugly Silverlight Duckling into a Beautiful Photoshopped Swan

This is a guest post by Samuel Jack, who had done a lot of work on the new UI for RavenDB.

Three weeks ago Ayende put out a request for help in turning an ugly duckling into a beautiful swan, and I, rather nervously, signed up for the job. The ugly duckling in question was Raven Studio, the Silverlight-based management UI for Raven Db. The nerves were a result of doubting that my limited graphic design skills were up to the job. But when Ayende assured me that he had a proto-type swan in the form of a Photoshop design, drawn up by a bona-fide, turtle-neck-wearing designer, they were calmed. Marginally.

Because the design Ayende had was for the new-look RavenDb website.

image

He wanted me to take the look and feel and transfer it to the Silverlight Raven Studio application. Which, when he handed it over to me, looked like this:

image

Ahh! Where to start?

Photoshop for Developer Dummies

To ease myself in, I got started by simply trying to imitate parts of the Photoshop design in XAML, beginning with the header bar at the very top of the page. Not being a designer myself, I’m rather like a duck out of water when it comes to Photoshop, but I’ve at least got the basics sussed.

The thing to understand is that designers construct Photoshop images like onions, layer upon layer, sometimes eye-watering in complexity, and to reproduce the design, you have to peel down through the layers.

Photoshop Layer BasicsFirst go to Photoshop’s Layers pane, and make sure all the layers are unlocked. This allows the Move Layer tool to come into play, not to move layers, but identify layers by selecting them in the Layers pane when you click the corresponding part of the image. Once you’ve identified a layer, Alt-Click it, and all other layers in the image will be hidden, allowing you to figure out exactly how the thing should look.

Mostly when I’m paring down Photoshop layers I’m looking to isolate them so that I can figure out the colour gradients they use. You could, of course, navigate your way through Photoshop’s dialogs to read off the exact RGB values. Or, if you can get the layer on its lonesome, you can use Expression Blend’s Gradient Dropper tool. GradientEyeDropper

This is a brilliant little timesaver. In the Blend Property pane, select the Brush property of your choice, put it into Gradient mode, click the Gradient Dropper tool, then drag over any area on screen, and Blend will reproduce the gradient under the cursor in XAML form.

After all that, I have the first feather of the new swan: a header bar matching the Photoshop design. Well, the background of the header bar. It needs fleshing out with some buttons.

Let it Live: Control Templates and Visual States

Silverlight, following WPF, has the concept of look-less controls. That is, the Controls (take Button as an example) manage their behaviour (Buttons respond to mouse clicks by executing commands) but don’t define how they are rendered on screen. That is left to the control’s Style, and specifically its ControlTemplate. The ControlTemplate defines the visual tree of all the UI elements needed to present the control and make it look snazzy. With a little patience, some assistance from Expression Blend, and plenty of application of the Gradient Dropper tool, it’s possible to take the built-in controls and make them look and feel just how the designer ordered.

I wanted Buttons that look like those in the header bar of the Photoshop design, but when the corresponding page is selected, they should change to have a background gradient with colours like the RavenDb logo.

When restyling a Control, it’s best to start by modifying the existing style. This way you can be sure you won’t miss an aspect of the control’s behaviour that you might otherwise forget. Blend makes this easy by giving you the option of editing a copy of the Control’s current ControlTemplate (right-click on it in the Objects and Timeline View, then select Edit Template > Edit a Copy). There are occasions when that little trick has failed, and I’ve ended up with an empty control template. But MSDN has come through for me then: it has a whole section containing the default Styles and Control Templates for all the built-in controls, like this one for Button.

Part of the ControlTemplate defines how the control looks in its various states, when the mouse is over it, when it has focus, or when it is selected, for example. The Control itself is responsible for indicating when it has entered each state. As a designer, it’s your job to specify Storyboards that are activated each time particular states are entered. Each Storyboard can animate whichever properties it likes to achieve the desired effect – in my Buttons, for example, I animate the opacity property of a Border element to fade in a coloured background indicating that it is selected. All this is overseen by the VisualStateManager, of which, more here. Naturally, Expression Blend has great editing support for visual states. Read John Papa’s tutorial to learn more.

So now I have a header bar with buttons that change colour when the corresponding page is selected. Where next?

Textured Backgrounds

Well, that page background could do with spicing up. The Photoshop design has a nice textured background, which I extracted to a PNG file that Silverlight would understand by hiding every layer except the background, then using Photoshop’s “Save for Web & Devices” feature.

The thing about textured backgrounds is that you do want them to cover the whole of the page background, which means tiling the texture to fill all the space. WPF makes this easy with its ImageBrush, which has a TileMode property, which, when set to a value other than None, automatically repeats the image over the whole area to be painted by the brush. Silverlight has ImageBrushes, but they don’t support tiling out of the box. Fortunately, Phil Middlemiss has supplied what is lacking in the form of the TiledBGControl which does exactly what I need – you should take a look: it makes clever use of Silverlight’s pixel shader effects.

This is what we’ve got so far.

image

The Index Editor Page, Before and After

Here are a couple of other pages I’ve beautified. First, the Index Editor page, as it was before:

image

And now:

image

Again, it was a challenge knowing where to apply my beautician’s brush first. I settled on adding the header bar at the top, and I then realised it could double up as a toolbar. Originally the page had no header at all, but by having a bread-crumb bar in the header it helps to give the user a bit more context when they’re looking at the page, as well as making it easier to navigate around.

Inspiration and Icons

Since my graphic-design skills are so underdeveloped, I borrow ideas shamelessly wherever I find something that fits. You may recognise the styling of the toolbar buttons in the Index page header bar as being remarkably similar to the ones on the Google Chrome toolbar. Yes – Expression Blend’s Gradient dropper does work on live applications too! Two places to check out if you find yourself short on inspiration are Quince and Pattern Tab which both catalogue examples of user interface and user experience design from across the web. Pattern Tab especially has myriad examples of beautiful UIs.

In the past I’ve struggled to find icons for my projects, but I’ve recently discovered two great sources: IconFinder.com and IconArchive.com. Both have excellent search facilities (which is often what’s missing from the commercial collections you buy and download to your machine in a whacking great zip file), and are careful to call out the license attached to each icon. A surprisingly large number are licensed so that they can be used without charge in commercial products.

A XAML Tip

The nice thing about styling an application is that it gets easier with every page you complete. Once you’ve settled on a look for a particular kind of element, you can repeat that look on every page. Silverlight’s support for Styles and Resources makes this very easy. And I have a tip that can make it easier still.

I put all my styles into a single resource dictionary, Styles.xaml which I merge into my App.xaml resource dictionary. I then name all my Styles, Brushes, etc. using a hierarchical naming convention. So Styles all begin with “Style_”, Styles for Buttons would all begin “Style_Button”, and then would come the styles for different purposes: “Style_Button_Toolbar”, “Style_Button_Hero” (for those big red buttons in your app that the hero uses to save the world), etc.. The pay-off for using this convention comes when you’re hand-editing XAML and making use of Resharper’s XAML intellisense. Type “{StaticResource Style_[ControlType]” and Resharper instantly presents you with a list of all the styles that might apply.

A Parting Screenshot

To finish, here’s one more before and after comparison, this time of the Edit Document page. Before:

Edit Document Page - Before

And after:

image

You can begin to sense the benefit of using a consistent set of styles, as it brings a harmonious feel to the whole application.

I hope you’ve enjoyed this whistle-stop tour of the Raven Studio beautification process. Remember that all the code is available on GitHub. We’d love to hear what you think.

Tags:

Published at

Originally posted at

Comments (2)

XAML Magic: Turning an Ugly Silverlight Duckling into a Beautiful Photoshopped Swan

This is a guest post by Samuel Jack, who had done a lot of work on the new UI for RavenDB.

Three weeks ago Ayende put out a request for help in turning an ugly duckling into a beautiful swan, and I, rather nervously, signed up for the job. The ugly duckling in question was Raven Studio, the Silverlight-based management UI for Raven Db. The nerves were a result of doubting that my limited graphic design skills were up to the job. But when Ayende assured me that he had a proto-type swan in the form of a Photoshop design, drawn up by a bona-fide, turtle-neck-wearing designer, they were calmed. Marginally.

Because the design Ayende had was for the new-look RavenDb website.

image

He wanted me to take the look and feel and transfer it to the Silverlight Raven Studio application. Which, when he handed it over to me, looked like this:

image

Ahh! Where to start?

Photoshop for Developer Dummies

To ease myself in, I got started by simply trying to imitate parts of the Photoshop design in XAML, beginning with the header bar at the very top of the page. Not being a designer myself, I’m rather like a duck out of water when it comes to Photoshop, but I’ve at least got the basics sussed.

The thing to understand is that designers construct Photoshop images like onions, layer upon layer, sometimes eye-watering in complexity, and to reproduce the design, you have to peel down through the layers.

Photoshop Layer BasicsFirst go to Photoshop’s Layers pane, and make sure all the layers are unlocked. This allows the Move Layer tool to come into play, not to move layers, but identify layers by selecting them in the Layers pane when you click the corresponding part of the image. Once you’ve identified a layer, Alt-Click it, and all other layers in the image will be hidden, allowing you to figure out exactly how the thing should look.

Mostly when I’m paring down Photoshop layers I’m looking to isolate them so that I can figure out the colour gradients they use. You could, of course, navigate your way through Photoshop’s dialogs to read off the exact RGB values. Or, if you can get the layer on its lonesome, you can use Expression Blend’s Gradient Dropper tool. GradientEyeDropper

This is a brilliant little timesaver. In the Blend Property pane, select the Brush property of your choice, put it into Gradient mode, click the Gradient Dropper tool, then drag over any area on screen, and Blend will reproduce the gradient under the cursor in XAML form.

After all that, I have the first feather of the new swan: a header bar matching the Photoshop design. Well, the background of the header bar. It needs fleshing out with some buttons.

Let it Live: Control Templates and Visual States

Silverlight, following WPF, has the concept of look-less controls. That is, the Controls (take Button as an example) manage their behaviour (Buttons respond to mouse clicks by executing commands) but don’t define how they are rendered on screen. That is left to the control’s Style, and specifically its ControlTemplate. The ControlTemplate defines the visual tree of all the UI elements needed to present the control and make it look snazzy. With a little patience, some assistance from Expression Blend, and plenty of application of the Gradient Dropper tool, it’s possible to take the built-in controls and make them look and feel just how the designer ordered.

I wanted Buttons that look like those in the header bar of the Photoshop design, but when the corresponding page is selected, they should change to have a background gradient with colours like the RavenDb logo.

When restyling a Control, it’s best to start by modifying the existing style. This way you can be sure you won’t miss an aspect of the control’s behaviour that you might otherwise forget. Blend makes this easy by giving you the option of editing a copy of the Control’s current ControlTemplate (right-click on it in the Objects and Timeline View, then select Edit Template > Edit a Copy). There are occasions when that little trick has failed, and I’ve ended up with an empty control template. But MSDN has come through for me then: it has a whole section containing the default Styles and Control Templates for all the built-in controls, like this one for Button.

Part of the ControlTemplate defines how the control looks in its various states, when the mouse is over it, when it has focus, or when it is selected, for example. The Control itself is responsible for indicating when it has entered each state. As a designer, it’s your job to specify Storyboards that are activated each time particular states are entered. Each Storyboard can animate whichever properties it likes to achieve the desired effect – in my Buttons, for example, I animate the opacity property of a Border element to fade in a coloured background indicating that it is selected. All this is overseen by the VisualStateManager, of which, more here. Naturally, Expression Blend has great editing support for visual states. Read John Papa’s tutorial to learn more.

So now I have a header bar with buttons that change colour when the corresponding page is selected. Where next?

Textured Backgrounds

Well, that page background could do with spicing up. The Photoshop design has a nice textured background, which I extracted to a PNG file that Silverlight would understand by hiding every layer except the background, then using Photoshop’s “Save for Web & Devices” feature.

The thing about textured backgrounds is that you do want them to cover the whole of the page background, which means tiling the texture to fill all the space. WPF makes this easy with its ImageBrush, which has a TileMode property, which, when set to a value other than None, automatically repeats the image over the whole area to be painted by the brush. Silverlight has ImageBrushes, but they don’t support tiling out of the box. Fortunately, Phil Middlemiss has supplied what is lacking in the form of the TiledBGControl which does exactly what I need – you should take a look: it makes clever use of Silverlight’s pixel shader effects.

This is what we’ve got so far.

image

The Index Editor Page, Before and After

Here are a couple of other pages I’ve beautified. First, the Index Editor page, as it was before:

image

And now:

image

Again, it was a challenge knowing where to apply my beautician’s brush first. I settled on adding the header bar at the top, and I then realised it could double up as a toolbar. Originally the page had no header at all, but by having a bread-crumb bar in the header it helps to give the user a bit more context when they’re looking at the page, as well as making it easier to navigate around.

Inspiration and Icons

Since my graphic-design skills are so underdeveloped, I borrow ideas shamelessly wherever I find something that fits. You may recognise the styling of the toolbar buttons in the Index page header bar as being remarkably similar to the ones on the Google Chrome toolbar. Yes – Expression Blend’s Gradient dropper does work on live applications too! Two places to check out if you find yourself short on inspiration are Quince and Pattern Tab which both catalogue examples of user interface and user experience design from across the web. Pattern Tab especially has myriad examples of beautiful UIs.

In the past I’ve struggled to find icons for my projects, but I’ve recently discovered two great sources: IconFinder.com and IconArchive.com. Both have excellent search facilities (which is often what’s missing from the commercial collections you buy and download to your machine in a whacking great zip file), and are careful to call out the license attached to each icon. A surprisingly large number are licensed so that they can be used without charge in commercial products.

A XAML Tip

The nice thing about styling an application is that it gets easier with every page you complete. Once you’ve settled on a look for a particular kind of element, you can repeat that look on every page. Silverlight’s support for Styles and Resources makes this very easy. And I have a tip that can make it easier still.

I put all my styles into a single resource dictionary, Styles.xaml which I merge into my App.xaml resource dictionary. I then name all my Styles, Brushes, etc. using a hierarchical naming convention. So Styles all begin with “Style_”, Styles for Buttons would all begin “Style_Button”, and then would come the styles for different purposes: “Style_Button_Toolbar”, “Style_Button_Hero” (for those big red buttons in your app that the hero uses to save the world), etc.. The pay-off for using this convention comes when you’re hand-editing XAML and making use of Resharper’s XAML intellisense. Type “{StaticResource Style_[ControlType]” and Resharper instantly presents you with a list of all the styles that might apply.

A Parting Screenshot

To finish, here’s one more before and after comparison, this time of the Edit Document page. Before:

Edit Document Page - Before

And after:

image

You can begin to sense the benefit of using a consistent set of styles, as it brings a harmonious feel to the whole application.

I hope you’ve enjoyed this whistle-stop tour of the Raven Studio beautification process. Remember that all the code is available on GitHub. We’d love to hear what you think.

Tags:

Published at

Originally posted at

Comments (2)