My Windows Phone cohort and all-round good guy Peter Torr posted on exactly the same topic that I did at almost exactly the same time... and we didn't co-ordinate it at all!

http://blogs.msdn.com/b/ptorr/archive/2010/08/16/virtualizing-data-in-windows-phone-7-silverlight-applications.aspx

Creepy, huh?

Peter has a great overview of virtualization and shows a different sample and given how tricky virtualization can be I highly recommend you also read his article.

There is a lot of data out there; on the the internet, tucked away in databases, sitting patiently on the other side of a REST web service just waiting to pounce on your unsuspecting Windows Phone application that just wants to display a little slice of it all so people can read it, touch it and generally make sense of it.  The problem is there is a lot of it, so what is a poor unsuspecting application to do, especially when it’s been crammed into a form factor that doesn’t allow ever expanding memory upgrades?

In this post and a few to come I’m going to explore the various ways you can work with the ListBox (the go-to control for displaying lists of data) in your application to keep it speedy and responsive.  Today we’re going to start with a feature added in Silverlight for Windows Phone: Data Virtualization.

Data Virtualization

Data virtualization is the concept of only loading “just enough” data to fill out your UI and be useful to interact with.  This is particularly useful if your data set is large and can’t all fit into memory at the same time.  Good examples are the 3,000 images you took of various pancakes shaped like childhood cartoon characters or all 23,083 remixes of The Postal Services “The District Sleeps Alone Tonight” you have loaded up on your phone.

Data virtualization in Silverlight is accomplished when you bind a custom IList implementation to a ListBox with a data template.  Let’s break that statement down by creating a very naïve virtualized list that simulates a list of 10,000 song objects.

Implement IList

This isn’t as daunting as it might seem, there are really only two methods you need to implement: Count and the indexer property.  Everything else can throw NotImplemented.  Count returns, well, the count of items while the indexer returns the actual item being requested by index.

Most of the magic happens in the indexer.  When an item is requested you now have a chance to go off and grab your data, fabricate it (such as using a WriteableBitmap), load it from IsolatedStorage, compute it based on index or generally return whatever makes sense.

Here I’m just creating a new Song class and setting its Title to a string that matches the requested index but in reality you’d be doing some parsing/loading/fetching here.

public class Song
{
    public string Title { get; set; }
    public string Length { get; set; }
}

public class VirtualSongList : IList<string>, IList
{
    /// <summary>
    /// Return the total number of items in your list.
    /// </summary>
    public int Count
    {
        get
        {
            return 10000;
        }
    }

    object IList.this[int index]
    {
        get
        {
            // here is where the magic happens, create/load your data on the fly.
            Debug.WriteLine("Requsted item " + index.ToString());
            return new Song() { Title = "Song " + index.ToString() };
        }
        set
        {
            throw new NotImplementedException();
        }
    }

    // everything else throws NotImplemented Exception.
    // .
    // .
    // .
}

NOTE: While the code is rather easy to implement it’s boring, mind numbingly boring so I’ve made the above implementation available for your source downloading pleasure.

Bind to ListBox

This is pretty straight-forward stuff, you’ll need to bind your new fancy list to the ListBox in question.

// Constructor
public MainPage()
{
    InitializeComponent();

    ItemList.ItemsSource = new VirtualSongList();
}

DataTemplate

One caveat is your ListBox needs to use a DataTemplate otherwise virtualization doesn’t kick in.  The virtualization code-path needs to make a few assumptions about your data and having a DataTemplate helps it down that path.  Without it all your hard work will go down the drain as the first time you bind your list every single item will be requested.

<ListBox x:Name="ItemList">
    <ListBox.ItemTemplate>
        <DataTemplate>
            <TextBlock Text="{Binding Title}" FontSize="32" />
	</DataTemplate>
    </ListBox.ItemTemplate>
</ListBox>

In Action

Let’s take this code for a spin.  An F5 and emulator launch later and we’re looking at this pretty screen:

As you can see we’re using the Song created during the indexer call.  To prove it’s actually virtualized I added a Debug.WriteLine to output every time an item was requested:

As you can see only 52 items were requested instead of the full 10,000.  Why 52 instead of just 13?  You always want a few items as buffer to give your virtualizing code a chance to actually do the work while items are being scrolled into view.  For this reason we request about three page worth of data so your UI is always responsive.

Extending And Testing

A virtualized list that only returns 10,000 items is a bit limited.  A better, more testable pattern would be to create a separate repository or “creator” class that actually creates, counts and manages the objects.  Your list should be nothing but that, a list.

Caching

Another thing to be aware of, and this is important to pay attention to, virtualization doesn’t do any caching for you.  This means if you scroll down and then back up your virtualized list will ask for index 0 again and if you have a naïve implementation you will recreate item 0 over and over again.   Depending on what you’re returning you may want your repository to have some kind of object cache based on either time or current index.

When To Virtualize

If you attempt to do a lot in your indexer’s get, where you actually make the “virtual” data real, you’ll soon discover that your UI is now *worse* than it was before. That’s because as you’re scrolling it’s getting hit over and over again so if you have an expensive creation you’ll block your UI thread and everything will come to a screaming to a halt (some people say screeching halt but if I can’t interact with my UI you will hear screaming).

The general rule of thumb on when to use data virtualization is for large lists or medium lists whose items are heavy weight, such as images and expensive XAML objects.  The bottom line is you just can’t have all 3,000 images loaded up at once so you need some way to give the user a feeling of a nice, continuous smooth scroll.

So how do you minimize the delay from heavy weight objects?  Stay tuned for my next blog post when I talk about using proxy (some people like to say broker) objects to push as much of the real work into the background thread.

I’ve noticed a few questions in the forums around why the Blur and DropShadow effects aren’t showing up in the Windows Phone 7 Series emulator and the simple answer is you have to set CacheMode to BitmapCache.

<TextBlock Text="DropShadow" Foreground="Black" FontSize="48" CacheMode="BitmapCache">
	<TextBlock.Effect>
		<DropShadowEffect/>
	</TextBlock.Effect>
</TextBlock>

<TextBlock Text="Blur" Foreground="Black" FontSize="48" CacheMode="BitmapCache">
	<TextBlock.Effect>
    	<BlurEffect/>
	</TextBlock.Effect>
</TextBlock>

We are working on setting this automatically so you don’t have to scratch your head each time you apply an effect and wonder why it looks fine on the design surface yet doesn’t show up in the emulator.

Given that the main way users enter text in your Windows Phone 7 Series application is via a tiny onscreen keyboard (a “Software Input Panel” or SIP) it’s in everyone’s best interest to make sure the right keys are available to the user when they need them.

A good example is when you go to enter a PIN number you really only want to see number-related keys,  when composing a SMS you’d like a quick way to insert an emoticon and if writing something longer you probably want auto-correction and replacement suggestions.  The good news for Windows Phone developers is you can control which SIP layout is used and which auto-correct settings are enabled by setting the InputScope property of your TextBox.

The concept of InputScope isn’t new to XAML by the way, this concept already exists in WPF and was a nice fit with the native SIP behavior so if you’ve used InputScope before you should be right at home.

Setting InputScope

There are really three ways to set InputScope, two of which are cumbersome yet provide Intellisense and a third that is easy but requires you to know exactly which InputScope you’re after.

Via XAML

Doing it this way you’ll get full Intellisense yet it’s a lot of work to set a simple property:

<TextBox>
    <TextBox.InputScope>
        <InputScope>
            <InputScopeName NameValue="Text" />
        </InputScope>
    </TextBox.InputScope>
</TextBox>

Via Code

If you’re more the code-behind sort here it is in C#:

textBox1.InputScope = new InputScope()
{
    Names = { new InputScopeName() { NameValue = InputScopeNameValue.Text } }
};

Via XAML with a TypeConvertor

Remember how in high school they’d always teach you the hard way before showing you the easy version? Guilty as charged.  If you already know the exact InputScopeNameValue you want to use, for example ‘Text’, then you can take advantage of the built-in TypeConvertor that is already wired up to the property and write this much easier XAML:

<TextBox InputScope="Text" />

InputScope Example Application

When we were bringing InputScopes from WPF to Silverlight for Windows Phone I wrote a quick app to show all the available InputScopes and automatically set the selected one on a TextBox.  This gives you a feel for how many there are and what layout comes up when set.  Here is what it looks like along with a link to the source:

Download the source.

Common InputScopes

Now that you know how to set them here are some of the more useful InputScopes:

Default	This is the default InputScope when no input scope is specified. Auto-capitalize first letter of sentence. The app can show app specific text suggestions. Layout: Standard QWERTY layout.
Number	When all you’re looking for is basic number entry. All features like auto-capitalize are turned off. Layout: the first symbol page of the standard QWERTY layout.
Text	When the user is typing standard text and can benefit from the full range of typing intelligence features: Text suggestions (while typing and when tapping on a word) Auto-correction Auto-Apostrophe (English) Auto-Accent Auto-capitalize first letter of sentence Layout: Text layout and access to letters, numbers, symbols and ASCII based emoticons + text suggestions. Example fields: email subject and body, OneNote notes, appointment subject and notes, Word document, etc.
Chat	The user is expected to type text using standard words as well as slang and abbreviations and can benefit from some of the typing intelligence features: Text suggestions (while typing and when tapping on a word) Auto-Apostrophe (English) Auto-Accent Auto-capitalize first letter of sentence Layout: Chat layout and access to letters, numbers, symbols and rich MSN like emoticons + text suggestions. Example fields: SMS, IM, Communicator, Twitter client, Facebook client, etc.
Url	The user is expected to type a URL. All auto-correct features are turned off. Layout: Web layout with “.com” and “go” key

For a complete list of InputScopes supported in Windows Phone 7 check out this MSDN link: InputScopeNameValue Enumeration.

In Conclusion

As you’re writing your applications don’t forget about InputScope, it’ll give it just that much more polish and can really make a difference in usability.

I promised someone at the end of my talk that I’d post my code and slides and while I’m lagging a little behind I’ve finally bundled them up and made them available for download here:

Slides + Code (YouAreHere.zip, named after the little app I built during the talk).

Also, as a small note I’m slowly working this blog over to have a “Windows Phone 7 Design Series” look and feel, not sure how well the dark background and light text is going to carry over or how well my design kung fu will stack up but here goes.

UPDATE #1: I’m also including the actual session itself here for your viewing pleasure.  You can either watch it below or directly on the MIX website.

UPDATE #2: Several people have asked me for sample code to the solution that has a Windows Phone and Silverlight desktop application both consuming the same Class Library.  Well, here it is for your downloading pleasure.

A quick heads up for anyone that saw my upcoming talk at MIX10.  I’m super excited (why do normal excited when you can be *super* excited!) to show everyone the basics of Silverlight so people can begin creating rockstar web and Windows Phone applications.  Speaking of the basics I want to share for anyone that did a Bing search wondering who this Shawn Oster fellow is and what his talk is really about.

My talk, An Introduction to Developing Applications for Microsoft Silverlight, really is an introduction, a 101.  Maybe you are a HTML/JavaScript ninja, perhaps you’re just starting out, maybe you decided hand tuning assembly had lost its luster, whatever reason if you’re a blank Silverlight slate then this is the talk for you.  On the other hand if you know about this crazy concept called XAML or have the basics of wiring up event handlers and doing a little data binding then this talk is definitely not for you.

If you’re hoping for some Windows Phone love then you don’t have to wait too much longer, Mike Harsh and Peter Torr lay down all the Windows Phone specific goods for you in the talks following mine.  Think of my talk as a primer for those that need a quick intro to some of the concepts they’ll be seeing later.

MIX is an awesome, information-packed developer/designer lovefest and I’d hate for you to miss out on another track having to sit through 101 information.

MIX is by far my favorite Microsoft event, and not just because it’s in Las Vegas.  It’s the mix of designers and developers, about learning cool stuff and just getting the creative juices flowing.  Whether I’m there in person or just watching the keynote via the web it has a rejuvenating effect on me and gets me excited to make cool stuff.

This year I’m also excited for two reasons: first it looks like I’ll get to speak (link to my talk to be posted once it’s on the schedule) and two because as a gadget and code nerd we get to learn about the development story of the upcoming Windows Phone!  Per the MIX website a few weeks back:

Yes, at MIX10 you’ll learn about developing applications and games for the next generation of Windows Phone. Yes, we’ll have Phone sessions, and we can’t say more…yet.

One thing I think people outside Microsoft don’t realize, at least I know I didn’t before working here, is that there is so much going on inside the company and we get so focused on whatever our project is that announcements like this are just as exciting for people inside the company as out!

A new Zune release, a new updated set of sources for Mp3tag.  The instructions are the same as always:

1. Download the latest Mp3tag Zune Marketplace Sources: ZuneMarketplace.zip
2. Unzip the two .src files into %appdata%\Mp3tag\data\sources
3. Click the Sources button in Mp3tag and you’re ready to rock!

Two notes about this update:

1. I renamed the sources to “Zune Marketplace” from just “Marketplace”, mostly to put it at the bottom of the list making it faster to select as well as giving Zune some branding love.
2. Now this change may be a bummer for some, I no longer have a good end-point for the 800x800 album art.  I used to be able to do a little dance and grab the nice hi-res album art but no longer.  I’m still poking around to see if I can’t find a better source but for now you’re stuck with 200x200.

One criticism we get as a company is that on the day Silverlight 1.0 was launched we didn’t do a massive refactoring of all Microsoft media-based or media-containing sites to use Silverlight.  Since dry humor doesn’t often translate via blog I’m poking some fun at our critics with that statement.  Any technology transition takes time, even one we believe strongly in like Silverlight.  Given skill sets, market requirements and different time-frames it was never that surprising that the previous Zune.net still used Flash.  It just made sense at the time.

Today is a different story though, today we can count one more site as having come into the Silverlight fold, Zune.net.  If you visit my Zune profile and do the right-click dance you’ll see the lovely “Silverlight” popup (circled in red below).

In fact Silverlight is used twice on the page, first as the portion that shows recent plays, badges and the artists a member is following as well as the media player on the bottom of the page.

That little player is pretty nice for playing 30-second previews but it’s even better when paired with a Zune Pass.  If you’re logged in and you have a Zune Pass then you can stream the full length track of anything in the Zune catalog.  You don’t even need the Zune client software installed to listen to full albums.  To give iTunes a friendly little dig you have install the software just to search their catalog.

Two great tastes together at last Zune and Silverlight.

The next generation of Zune is here and as someone that has been fortunate enough to play with both the new software and a ZuneHD I wanted to share some of my favorite things in the new software.  More details on the ZuneHD itself to come, as while I wrote this post the software features kept piling up so I’ll save the hardware focused one for tomorrow.

Quickplay

This is the new landing page when you first open the software and you’re instantly given a rich, media-centric view that shows new items, your most recently listened to items as well as anything you’ve pinned to the Quickplay menu.  One of my favorite things about the Zune software is the fact that the UI adds to the media experience instead of just being a database of media.

While Quickplay is a nice, big, in-your-face feature there are quite a few little details I’d like to call out that you may miss.

Smart DJ

OK, so this isn’t really a subtle, little detail and many will argue that this is the feature that makes Zune 4.0 worthy.  Smart DJ is like the iTunes Genius feature married to Pandora, able to create a never-ending stream of music based on an artist or song.  Where this really shines is when you couple it with a Zune Pass because now you can pull music not just from your collection but from the entire Zune catalog.  I’ve already discovered a ton of great music this way with the added benefit of being able to save these Smart DJ recommendations to playlists that I can then put on my device.

Remaining Credits & Expiration Date

Your Zune Pass gives you 10 free songs month but they don’t roll over so it’s nice to know when you’re entering “use ‘em or lose ‘em” territory. Now as their expiration date grows close the number of credits you have left turns a bolded pink and the credit’s expiration date is shown, calling out that you better use them soon.  Given that I’m not close to my date it’s hard to screen capture this feature :)

Suggested Songs

If you click on your remaining credits you’ll be offered a list of suggested songs you might want to own, another great feature because invariably right when you need to pick exactly 10 songs is when you completely blank on all the music you’ve listened to, ever, and you end up either wasting the free songs or buying something you didn’t really want.

Content Filtering

If you’re a Zune Pass subscriber you’ve probably found yourself asking, “what music do I actually own and what is from my Zune Pass?”  Wonder no more with the ability to filter your content:

New Backgrounds

Silly as it is every new release I look to see what new backgrounds are available and they didn’t disappoint, my favorite new one is Geisha.

Windows 7 Support

AeroSnap

One of the best UX features in Windows 7 to me is AeroSnap, the ability to use Win+Left/Right Arrow to automatically position a window at 50% of your screen.  The Zune 3.0 software didn’t work with it, 4.0 gives us the snap love.

Taskbar Thumbnails

Full support for taskbar thumbnails, including a nice way to quickly pause and rate the current song.

Jump Lists

Another great Windows 7 feature that you slowly come to depend on is Jump Lists, now fully supported with Zune 4.0, offering quick access to your Quickplay pinned items and Smart DJ lists.

New Now Playing Layout

Another thing I’ve always loved about the Zune software is that instead of going the 1980’s route of WinAmp-style visualizations (sorry candy kids) it went for an artistic mashup of colors, art and metadata.  That combined with some typographic treatment has really made the Now Playing screen one of my favorites in the software and something that I’d love to see on my 50” plasma during a cocktail party vs. the seizure-inducing XBox or WinAmp fractural crackfest.

Here is Now Playing when there isn’t any hi-res art to use.  Previously all the album covers were the same size, with 4.0 they’re varied giving the UI a more organic feel.

Whew, and that’s just off the top of my head while I’m waiting for my bus :)  Needless to say the Zune 4.0 brings a host of new features and improvements on existing ones that makes this release exciting even without the ZuneHD.  Of course the ZuneHD is here and as I’ll blog about later it’s a beauty.