What is so difficult about broadcasting MLB.TV?
As a member of the development team for the Major League Baseball MLB.TV project, I've had a somewhat unique perspective on the successes and failures of the project. There is a bit of backlash in the blogosphere about the technologies surrounding the players, and the inherent superiority of one technology over the other. This post examines only the technologies involved in the situation, and not any of the underlying business decisions surrounding it.
So, what is so difficult about broadcasting MLB.TV? Funny you should ask, regardless of all the Flash vs Silverlight debate that has surrounded the MLB.TV launch this year, the player itself represents only about 10% of the complexity. So, how does it all work? There are a series of inter-related processes involved.
- Acquisition of feeds – Home and away video feeds from the ball park are provided by the networks covering the games. If there are problems with those feeds, the video/audio never makes it to MLB. As an example of this, in one of the opening games, an unplugged cable in the feed truck was responsible for lack of an audio feed in the MLB.TV player. There are any number of things which could go wrong at the ball park or in the broadcast trucks to provide the signal. Assuming all is acquired properly, the next step is...
- Encoding of feeds – The video and audio content streams from the broadcasters at the ballpark across the network to the MLB Advanced Media offices in New York City. There, the data is encoded into various formats for the various players, including MLB.TV, Gameday Audio, iPhone, etc. For MLB.TV, the stream is encoded at 7 different bit-rates, so the proper quality of video can be delivered, based on the end users connection speed to the internet.
- Provisioning – As a game starts, all of the data about the game and the feeds available for it needs to be provisioned into internal systems at MLB. This data is used by all the various applications to determine what game's are available, what feeds are available for each game, and which qualities are available for each feed. If one of the encoders fails, or one of the feeds drops, or anything else goes wrong in steps 1 and 2, the data needs to be re-provisioned to reflect the current accurate data for a game.
- Services - There are a series of Java based services which consume the provisioned data, and make it available to the player and other applications. Amongst the jobs the services provide are login, authentication, geo detection (to help enforce blackout restrictions), managing stream security and more. Of course, these services are dependent on the data about each game being available and accurate.
- Streaming Servers – There are dozens of Flash Media Servers at the Content Delivery Network (CDN) partner of MLB, from which all the video and audio (and lots of other things too) are streamed to the end user. The CDN has the capacity to stream hundreds of thousands of games at once.
- MLB.TV player – While this is the only part of the process most end users see, this is merely a front end to all the systems described above. While we are constantly finding problems, fixing them, making improvements, testing to ensure we haven't caused other problems, 99% of all bugs reported in the Forums and MLBlog actually relate to one of the "upstream" elements of this system. As I mentioned earlier, an unplugged cable caused hundreds to complain that our flash version of the media player couldn't properly play sounds. Obviously, that wasn't the problem, as all the other games, and even the other feed for that game properly had the audio channels playing, it's simply a matter of the player playing the content which was provided to it.
All told, I'm honored to be a part of the team which made this all possible, am proud that the vast majority of users are able to view high quality content of the games as they are played, and will work tirelessly to help fix the problems reported by the others.


Thanks for posting this, it really goes to show people how much really has to go right to get a feed from the field onto our PC's. Thanks for the hard work and I'm confident things will work out soon.
Also, there are infinite complaints about the service from the MLB end, including the whole idea of charging people extra for the premium service that is not delivered. Both sides are pretty terrible here.
thousands of customers since the regular season began and MLB Support's complete
refusal to address the issue? Thousands have reported choppy video when using
NextDef and nothing is being told to any of us despite the fact that we paid extra
money for the premium service you advertised. The support's laughable response has
been to reboot your computer and clear cache. Please start taking the problems of
taking paying customers seriously and then we'll stop "bludgeoning" MLB.tv. All we want
is it to work!
Like I said... 2 games into the season and I haven't been able to watch a full inning without failures.
Enjoy this article and know that most of the MLB.TV subscribers experienced the same and feel the same way as the author.
http://bleacherreport.com/articles/152306-mlbtv-st....
I won't be offended in you not posting this response. I just had to inform you about the truth.
Thanks,
Laurent
So, tell us what business decisions made you go from Silverlight to Flash ;-)
NexDef actually intermingles at a few different places here, including a separate set of encoders, a different set of streams from the CDN, and a different process running on the individual users machines. For NexDef, effectivly, the flash player asks the local process for the content, and that local process manages requesting the appropriate streams from the CDN, allowing for it to pause, rewind, etc.
If you see my comment to Steind, I give a bit of insight into what the NexDef process is. That too has a number of places where things can go wrong. But believe me that all the various teams are continuing to work to make the experience better.
I dont make any excuse for anything that has happened. It may come as news to you, but software always has bugs, and sometimes doesnt work right. Bugs in software is not unique to MLB.TV, Flash, SilverLight, Windows, MacOS, Linux, or any other piece of software you may have ever used. We, as software developers continue to try and fix the bugs that exist in our software, while trying not to introduce new ones.
However, even if there arent any bugs whatsoever (as unlikely as that may be), I'm trying to point out in this article all the various things that can still go wrong in the process. So, specifically, poor quality could be a result of:
1) Bad Feeds From the Ball Park
2) Excessive Internet Traffic Between Ballpark and Encoders
3) Bad Encoding
4) Excessive Internet Traffic between encoders and CDN
5) Problems at CDN
6) Excessive Internet Traffic between CDN and your location
7) Slow internet connection on your machine
8) Other software on your machine running slowly, so the player doesn't have adequate CPU to run
probably a bunch of other things i cant think of too.
Anyhow, no excuses, but from your post I'm not sure that you realize that the vast majority of users are watching games daily with very few issues? I'm personally sorry ( I'm not in a position to apologize on MLB's behalf ) that you have issues with the player.
As frustrating as it is to hear though, you should publish any specifics about these issues on the Official MLB Support Forums, so they can be reviewed and added to our official bug queue. My blog is not an official support channel, and while I take things posted here seriously, I doubt "the powers that be" are looking here for bug reports.
im as much for poking at others insecurities as the next guy, but I prefer to keep it on a personal level. So whats up with that shirt?
No seriously, it wouldnt take any one reading this more than a few clicks to figure out where i sit in the whole flash v silverlight thing, but was trying to get above the fray on that one for once.
It is great to hear from folks who are having a good experience. I'm lead to believe that this is the case for the majority of users, however, most for whom it just works arent posting much about it online, resulting in lots of posts about the problems and few about what does work. Not surprising, thats just the way the world works.
Anyhow, thanks for posting, and glad you are enjoying it
Even if I knew that answer, I couldnt tell you. All I really know though is my company (Digital Primates) was hired to help with the Flash front end on this project. By the time we were hired, MLB had already made the decision to switch from Silverlight.
Honestly, we've been involved in a number of projects like this where a client was switching from SilverLight (or AJAX, or JavaFX, etc.) to Flash, just like many former clients of ours have switched from Flash to one of the other technologies. Each technology has a different place where they excel, so there is no one right answer for every project. (Jesse, am i being too technologically politically correct again?)
Good luck!
jeff
So, tell us what business decisions made you go from Silverlight to Flash ;-)
http://www.xspyz.com