Well it turns out Twitter just turned off allowing anyone from accessing their site via cross domain requests because of a security hole that was found.  It sounds like they're going to be doing some good things to enable access to their api through cross domain requests but it's going to take a little while.  For the time being I'll probably add a proxy to my own site so I can continue to play around with Silverlight.  It is pretty amazing that literally the day that I try and use Twitter's api via Silverlight they turned off outside access.  For those interested they're going to be doing the following within the next month:

1. Move the Twitter API to api.twitter.com.  Use the completely permissive crossdomain.xml on api.twitter.com.

2. Stop supporting HTTP Basic auth. on api.twitter.com.  Implement OAuth or some other kind of auth. token system.

3. Require non-public API requests to include a valid user auth. token.

All three items were recommended by some chap named Dossy on the Twitter Development Google Talk Group.  On a side note OAuth seems to be gaining some traction and definitely needs some more of my attention.