This week we have a story to add the ability to extract some of the data within our system into Excel format.  Upon initially hearing the requirement I thought of Visual Studio Tools for Office (VSTO), but after digging into the documentation for a bit, it doesn't seem that it was meant for our scenario.  VSTO seems to be ideal for Office Add-ins but quite lacking in its support for server based scenarios.

We did a bit of research and found a couple commercial options such as Software Artisans OfficeWriter and Aspose.Cells. Additionally, this evening Aaron emailed me about Carlos Aguiar's ExcelXmlWriter project that provides a free utility for generating Excel files as well as a nice generator. 

I'd certainly be interested in hearing about any other components that anyone has used for generating Excel in a server environment and/or any experiences with the products mentioned above.

My primary machine at home is a MacBook Pro.  Overall I really like it as my primary machine.  When I first got my MBP I used it almost exclusively to run Windows XP via BootCamp.  It wasn't until I tried using a beta of Parallels that promised BootCamp support that I ended up switching to Parallels and the wonder that is Coherence mode.  Over the last couple of months I've been doing mostly writing and/or work in TextMate so I've used XP via Parallels very little.  Lately I've had a couple of things come up that have me back in XP with VS.NET and some non book sample sized projects.  I'm finding it quite painful.  Most of my usual keyboard shortcuts don't work (althought I can get some to work by using the fn key), things seem painfully slow lately, and I'm not a happy camper.  I'm trying to figure out if I need to reconfigure my Parallels image, or if its time to switch back to BootCamp.

Today at work we had a brief discussion about whether using an IN or an EXISTS is more performant.  There were a couple different opinions, although most thought that EXISTS was faster.  It turns out that like most things in our wonderful software world there is not clear cut answer.  In general the below holds true:

• If the majority of the filtering criteria are in the subquery then the IN variation may be more performant.
• If the majority of the filtering criteria are in the top query then the EXISTS variation may be more performant.

Source: http://www.oracle-base.com/articles/misc/EfficientSQLStatements.php#Exists and this.

I assume the same holds true for SQL Server, although we're less interested in that at the moment.

A nice UI does wonders for an application.  As such if you're a Google Reader user, I recommend you checkout this Google Reader theme.

Mohammed Hossam El-Din has created an implementation of IQueryable that enables LINQ to be used against Flickr.  The LINQ to Flickr implementation leverages the Flickr.NET library.  I've been wanting to write my own implementation of IQueryable but it looks like everyone is beating me to all the good data sources. :)  Hopefully, once I finish my chapters for the LINQ book I'll have a chance to complete an implementation of IQueryable for an interesting data source...if there are any left  :)

Bart De Smart has a great series of posts on how to create a custom implemention of IQueryable that enables LINQ support for LDAP.

Over the last week or so I've been following a thread of blog posts that outline some of the issues that Twitter is experiencing.  For those of unfamiliar, Twitter is a "microblogging" site written in Ruby on Rails.  Over the last month or so they've seen some very serious growth.  They're supposedly seeing over 11,000 request/sec which is putting a hurting on their servers.  In this interview, Alex Payne points out some limitations in Rails, most notably it's lack of built in support for talking to multiple databases, that are making it particularly hard for them to handle the increased load.  As a result, several individual have put together plugins to make working with multiple read only slave databases possible.

• Magic Multi-Connections: A “facility in Rails to talk to more than one database at a time”
• ActsAsReadonlyable

Along with checking out the plugins mentioned above you may want to checkout the following posts that go into more detail on the problems.

• Twitter trouble
• Twitter, Ruby, and Scaling
• Rails and Scaling with Multiple Databases

What I find particularly interesting in the series of posts that talk about the scaling problems that Twitter is experiencing is that most of their problems are arising due to the database server getting nailed.  They've been able to scale out their web tier but that doesn't help their poor database server in the least.  I've been thinking about writing a series (or maybe just one) post on some of the things we've been thinking about regarding scaling our application.  It's not written in Rails, and it doesn't need to handle 11,000 req/sec but it does need to handle large sets of data and be "hot ass fast". We're currently positioned well to handle scaling out our services tier quite easily, but we need to do some more work to figure out how best to scale out our data tier.  We've had many discussions about how scaling out our services may not help since it will only increase the pressure on the database server.  Anyway, hopefully I'll get a chance to post more about the things that we're thinking about in a future post.

Jumping back to the Twitter/Rails scaling problems...I'll be interested to see how things end up with Twitter, and Rails.  I have a feeling that Rails will be better off when all is said and done, but will it be because of Twitter, or in spite of it?

Recently we've been discussing ways to increase the performance of some performance critical areas of our application.  As such, a number of individuals have been doing research into various options.  Through our investigation we came upon Oracle's Timesten In-Memory database.  I was familiar with many other In-Memory databases, such as Firebird (SQLite?, VistaDB?), but up until yesterday I wasn't aware that Oracle had a product in the in-memory space.  While Oracle doesn't currently offer a .NET native provider for Timesten, TankardSoft does offer a commercial native .NET data provider for Timesten that supports the recently released version 7 of Timesten. 

If you're an NHibernate user, and/or interesting in learning about how you can create custom implementations of LINQ using IQueryable, I strongly urge you to checkout what Ayende and Bobby Diaz (and others?) have been up to with LINQ to NHibernate.

Here are some of my favorite posts thus far:

• Implmenting Linq for NHibernate: A How to Guide Part 1
  
• Implementing Linq for NHibernate - Part 2 (Ordering and Paging)
• Implementing Linq for NHibernate - Part 3 (Aggregate and Element Operators)