C# 4.0, Dynamic Programming and JSON

C# 4.0 will introduce dynamic programming and late-bound code in the midst of otherwise statically compiled code... in an extensible way. Heres one scenario enhanced with dynamic capabilities - working with JSON data.

C# 4.0 features a new dynamic keyword that allows you to mix in a bit of late-bound code in the midst of your otherwise statically typed code. This helps cleanup the string-based programming mess that is a characteristic of late-bound code. In fact, there are a number of scenarios that would benefit from dynamic typing in my opinion in addition to interop with other dynamic code (such as Silverlight talking to the DOM or a .NET app talking to Office automation APIs). For example:

  • Accessing JSON data
  • Accessing XML nodes and attributes
  • Experimenting and calling into with REST services without an explicitly coded up proxy
  • Access to settings (eg. Isolated storage settings in Silverlight, or configuration settings such as appSettings)
  • ... and more

Traditional dynamic languages are striving to introduce some degree of static typing (eg. type information in ActionScript 3, and the unfortunately failed EcmaScript 4 attempt) for perf and more appeal. On the flip side, C# is evolving nicely by introducing dynamic typing through a static type called dynamic (nice oxymoron) and offering not just a nicer syntax, but a much more intuitive model for these scenarios. I can only wish the C# 4.0 was here now!

Anyway, I am still excited about this feature, and given the VS 2010 and C# 4.0 CTP are available, I decided to play with a few of the above listed scenarios. In this post, I'll describe the JSON scenario, and in a subsequent post, I'll write about the REST services scenario, so stay tuned and come back for more.

First a quick look at how JSON data might be used in the framework today, specifically using System.Json in Silverlight.

string jsonText = "{ xyz: 123, items: [ 10, 100, 1000 ] }";
JsonObject jsonObject = (JsonObject)JsonValue.Parse(jsonText);

JsonArray items = (JsonArray)jsonObject["items"];
items[2] = 1001;

JsonObject bar = new JsonObject();
bar["name"] = "c#";
jsonObject["bar"] = bar; 

StringWriter sw = new StringWriter();
jsonObject.Save(sw);
string newJsonText = sw.ToString();

Contrast this with using dynamic typing. I basically took my very light-weight JSON reader and writer and added support for dynamic typing in my equivalent JsonObject and JsonArray types. With that in place, I can now write code as follows:

string jsonText = "{ xyz: 123, items: [ 10, 100, 1000 ] }";

JsonReader jsonReader = new JsonReader(jsonText);
dynamic jsonObject = jsonReader.ReadValue();

dynamic items = jsonObject.items;
items.Item(2, 1001);

dynamic bar = new JsonObject();
bar.name = "c#";
jsonObject.bar = bar;

JsonWriter writer = new JsonWriter();
writer.WriteValue((object)jsonObject);

string newJsonText = writer.Json;

Note that in the CTP, there isn't any support for indexers used against dynamic types, which gets in the way of normal array syntax. Hence the workaround above using Item(). However, I've been told, that support for indexing into dynamic types already exists in later builds.

Personally, I think the resulting code using dynamic typing is nicer than the current System.Json APIs, since you don't have to use strings for member names, and that distinguishes them from actual string values. Thoughts?

If you've got the CTP, you can download the project and play with it yourself.

Even if you don't have the CTP, it is fairly straightforward to see whats going on. Here is a quick guide to the code: Basically you'll see a couple of classes JsonObject and JsonArray - both of these derive from DynamicObject, which itself implements IDynamicObject, the entry-point to building a type that supports late-bound programming. JsonReader and JsonWriter itself implement a light-weight JSON parser and formatter.

Also, the PDC sessions on C# 4.0 and dynamic languages are both interesting sessions to check out if you're interested in this space.


[ Tags: | | ]
Posted on Monday, 11/3/2008 @ 7:18 AM | #Projects


Comments

27 comments have been posted.

Francois Ward

Posted on 11/3/2008 @ 8:41 AM
I heavily dislike this. My philosophy has always been "right tool for the right job". There are dynamic languages that are vastly superior to C#, if you want dynamic. C# was meant to be a "pure" language, to do the stuff where you want as much strong typing as possible (also the reason behind Spec#), and as clean as possible.

This defeats that purpose in such a way that only an FxCop rule or code reviews could stop it from ruinning a codebase. If I wanted dynamic, I could do it in IronRuby (once thats fully out), and call the result from C#.

More so: this dynamic feature was mostly meant to help with COM interop, not as a convenience to save a few lines of code... and people are -already- thinking of ways to misuse it... Really, the .NET runtime was made so we could have all the languages we want on it, EXACTLY so we wouldn't need a "one language to rule them all" thing... The best codebases would use C# as the core, IronRuby (or something similar) for places where you need Dynamic, and F# for places where you need functional... there's no need to stick C# everywhere...and thus, there's no need to add this to C#. Its too late now, but at the very least we can make sure its not abused.

Anshu Bansal

Posted on 11/3/2008 @ 9:25 AM
Hi,
I fairly dislike that. I don't understand what Microsoft is trying to achieve with this. Instead of getting on simple path they are opting a complex path. If you go for this technology you need to remember complicated syntax by heart. No body can understand Microsoft's philosophy. They don't fully develop a working product but keep launching new and new products every year. They launch buggy and beta products but never try to develop a full fledget working product and they improve it.

Chris

Posted on 11/3/2008 @ 10:19 AM
I agree with the above. .NET has always been reletavely clean for an enterprise scale framework. This really isnt needed in C#, we're just making the whole thing a little messier.

Nikhil Kothari

Posted on 11/3/2008 @ 11:08 AM
Certainly I don't think anyone is suggesting going whole hog dynamic or late-bound as opposed to statically typed code - its a tool that should be used carefully just like many other existing C# features (yield, var, anonymous types etc. etc.), as well as existing framework features that already require varying degrees of late-binding (such as reflection, or even something like a DataReader). In many ways what is new is more natural syntax from a language perspective, and a mechanism to do so consistently across different late-binding scenarios.

Who better to reference than Anders. As he said in his talk, the reality is both dynamic and static languages have their virtues, and the reality is you want the good attributes of both in your toolset. Definitely check out the first of the PDC sessions I linked to - about at about the 6:30 mark and the 14:30 mark as well. Even Anders at around 20:45 mark talks about the virtue of static typing but where things not statically typed can be made easier, as opposed to painful. Hope that helps position this new feature...

Joe

Posted on 11/4/2008 @ 4:15 AM
Don't do it.

Miguel

Posted on 11/4/2008 @ 4:40 AM
I don't see how this
something.Property = "";
provides a big advantage over:
something["Property"] = "";

The syntax is better, but it's not a big dfifference. I can see some benefits of using the dynamic's features of C#, but I don't think this is one of them. Something similar with the datareader example:
o = reader["Column"];
or
o = reader.Column;

Anyway, the argument the syntax is better is good enough for me, so what does it mean? We're exposed to similar late-bound/runtime risks, we still know where they are, but we can have cleaner code. I'm sure the tools might even be able to detect references to the same properties, do renames and stuff like that, so that's something we don't have on dictories or lets say datareaders. So the previous implementation of JSON Object gave me the same late-bound access using dictonaries, now we can choose to use a better syntax (actually JS is implemented on top of dictionaries and when we use object.Property is just syntactic sugar).


I agree with Francois on the Ruby aspect, however, I think that having some functional features in C# was one of the greatest things that happen to the langguage after generics (not that a lot has happened), but on it's moment many people were saying, "why do we need lambdas if we could use F# and the syntax is horrible", so we might be wrong and this might evolve to a point where we actually start using it well.

I just tought about another escenario where this might be useful. XAML already works on a latebound way so you might code this in C#:
LayoutRoot.DataContext = jasonObject;
and in XAML have something like:
<TextBox Text={Binding Name}/>
and the Name property will be determined by the Dynamic Object at runtime based on whatever string it was constructed with. Otherwise we would need to create another class just for this task since XAML can't use indexers. Another implementation might be with ValueConverters
<TextBox Text={Binding Path=jasonObject, Converter={{StaticResource JasonObjectConverter}, ConverterParameter=Name}}/>

Nikhil Kothari

Posted on 11/4/2008 @ 9:54 AM
@Miguel - you pointed out tools might even be able to detect references to the same properties...

I imagine tooling for dynamic programming will in fact improve over time.

Furthermore, just noting in code that something is dynamic provides more information to tools that are trying to analyze code than just string names. And hopefully such analysis will be able to run regardless of which late-binding mechanism is being abstracted - whether its reflection, DLR interop, DataReader, JSON etc. etc. whereas previously a tool might only be able to understand one particular scenario.

Bertrand Le Roy

Posted on 11/4/2008 @ 1:36 PM
We see this kind of misunderstanding with every new version. The 'var' keyword, which is a necessary piece of the Linq picture, got similar criticism. Anders said it best in his talk: C# was never meant to be a "pure" language. It was meant to be productive. The reflection scenarios alone justify this feature. Anyone who has had to use reflection heavily should be happy about this.

Travis

Posted on 11/4/2008 @ 2:38 PM
Using "var" and "dynamic" in the same argument is simply laughable. They are not related at all.

Nikhil Kothari

Posted on 11/4/2008 @ 10:16 PM
@Travis - I am pretty sure Bertrand is using "var" as an example of a particular c# language feature that has its uses when used in appropriate contexts... rather than relating them in terms of functionality.

Shafqat Ahmed

Posted on 11/5/2008 @ 11:56 AM
The problem is when people write bad code this becomes a bad feature. For example multiple inheritance in C++ is awesome but many people could not use it and that is why we have single inheritance in .NET and Java. We do not need to make C# a dynamic language, there are languages that do it just fine. This will make a lot of newbies experiment and want to use this feature in their code resulting in bad architecture.

Nikhil Kothari

Posted on 11/5/2008 @ 12:11 PM
I am sure as time goes by (once the dynamic feature is actually available in release form), a set of guidelines will develop around it, for what is good use, and what isn't. Like any new feature, whether its language or framework, it will get its fair share of early experimentation, some extreme positions will indeed develop (theres probably two camps on these sort of things), and over time more pragmatic use will become mainstream.

I actually look at things from a diametrically opposite perspective. If I want/need a little bit of dynamic programming, I don't want to completely switch to a different language, and in doing so lose all static typing for all of my code, or create artificial lines right through what ought to be a single component or application to separate the two halves. Instead I want to keep most of my code statically typed, encapsulate the dynamic bits as implementation detail of some statically typed object that is then used elsewhere in the overall body of code. So personally, I love the dynamic feature as another tool in the toolbox for use when the situation seems appropriate and its use is justified.

That said, interesting commentary! It is however odd that I am the one trying to justify dynamic programming, when I also passionately believe in script#, whose core feature is static typing and compilation of c# down to script. Honestly, I don't think things are that black and white... theres good uses of dynamic programming especially, when you can contain it, and easily identify it (like the syntax in c# allows you to do so).

Francois Ward

Posted on 11/5/2008 @ 5:18 PM
My original point really was that this goes against the "pit of success" philosophy: now, to make sure things like this don't get abused, I need FxCop rules, and they're going to be harder to add than for var.

That said... maybe C# wasn't meant to be a pure language...but it very much so was one... and virtually everyone I knew who switched from VB to C# (the latest stats showed that C# really gained over VB now... there has to be a reason) did it because of that... and the third party tooling was better for C#, too, and not just because people making the tools liked C# better :) If C# is supposed to become the kitchen sink language, so be it... but in that case, can we get another language to replace it? To fit the gap that soon it will not fit anymore? An heavily supported, heavily static, no bells and whistles (ok, you can leave LINQ and lambdas :) ) language with a narrow pit of success? Then the rest can do whatever they wish with C# and everyone will be happy =)

Really, now that I have to read and maintain code that will be marbled with late bound magic all over... is there -ANY- reason not to simply switch to IronRuby once its out? At least then I'll -expect- the dynamic stuff, and the language handles the mess better. So what niche does C# fill anymore?

Too bad there's no time machine... it would be interested to see... C# going down the path its going, and a brand new (but just as supported) purist language that wouldn't go too much beyond the feature set of 3.5 and some polish...and see what happens... though in my opinion, we already saw that scenario, back in the early 2000s...

Miguel

Posted on 11/6/2008 @ 5:21 AM
Yes, switching to IronRuby will be an option and I have to say, a nice one, from a language perspective you can see this as just more syntactic sugar like extension methods, so no point in doing a quite complex class that takes strings and returns objects that take other strings and return other objects.
var something = dynamicObject["Something"];
(something["GetFunction"] as Func)();
(something["Event"] as EventHandler)+= (something["GetOtherFunction"] as Func<object,EventArts>)

Then look at this:
dynamic something = dynamictObject.Something;
something.Function()
something.Event+= something.OtherFunction;

Just like the JSON class before, it's working know, but it's hard to use, so now we will have a nicer syntax to use.

Chris

Posted on 11/12/2008 @ 4:48 AM
I agree with the title of this post. I like it. I think it IS valuable for those of us who know that JSON is the future protocol between clients and servers. Let's face it, HTML is too static, XML is too wordy.

Keep up the good work Nikhil.

Bryan

Posted on 11/14/2008 @ 6:12 AM
Extension methods aren't just syntactic sugar. They are already being abused and causing problems. See Linq and the MVC framework for rampant examples of how the dynamic functionality will be abused once it is released.

Nikhil Kothari

Posted on 11/15/2008 @ 3:15 AM
Forward reference to the followup blog post on using dynamic to implement REST service access:
http://www.nikhilk.net/CSharp-Dynamic-Programming-REST-Services.aspx

jeremy simmons

Posted on 11/18/2008 @ 8:29 AM
It's like any other tool. Don't use it everywhere, only where appropriate.
You wouldn't use screws on baseboard, or finish nails on decking.
Just like a carpenter, software engineers must refine their trade and recognize the right tool for the job.

The_Assimilator

Posted on 11/20/2008 @ 12:04 AM
Having worked extensively with late-bound scenarios (.NET 1.1 data binding and of course Reflection), I have to say this feature looks very promising. Certainly it will make late-bound code much easier to write and read, and if that isn't a hallmark of a good language I don't know what is.

As for the guys complaining that this feature will be misused - it takes a bad programmer to write bad code, and in my experience bad programmers manage to screw up any language feature they come across. Don't blame the language for its users' mistakes.

Joseph

Posted on 12/5/2008 @ 12:58 PM
Is it possible to use the compiled RestClient binary in a .net 3.5 app or is dynamic keyword/feature tied to the .net 4 runtime?

HB

Posted on 12/9/2008 @ 11:25 AM
Why such resistance to this? Mandate to your team not to use it if you don't like it.

Every release of every programming language has the same problem. People resist the new features claiming that they will be abused and ruin everything and yet here we are in C# 4.0 and the use of 'var', anonymous <fill in the blank> and other C# 3.0 features haven't destroyed us.

For people that use a lot of Json (like in MVC), this will be especially handy

HB

Posted on 12/9/2008 @ 11:35 AM
... I was thinking extension methods there, but it didn't come to mind...

MCE

Posted on 12/11/2008 @ 9:16 AM
There isn't anything new . This feaure has always been supported in VB.NET. This is copy paste work. There is not value on this. They are destroying the language.

Having worked extensively with late-bound scenarios (.NET 1.1 data binding and of course Reflection), I have to say this feature looks very promising. Certainly it will make late-bound code much easier to write and read, and if that isn't a hallmark of a good language I don't know what is.

If that was your case you could wrap your object in a VB.NET dll and call them from C#. There isn't any value or Innovation on this COPY PASTE work.

This is a sad ending for a great language It is so sad.

Nikhil Kothari

Posted on 12/19/2008 @ 12:15 AM
MCE - This is not the same as late-binding that already exists in VB.NET - VB's existing binder works on top of things like reflection and I believe OLE automation/IDispatch as well. It is not generic or extensible to allow mapping late-bound calls into a more domain-specific API or infrastructure - certainly not something as different as mapping late-bound calls to HTTP requests.

I can imagine VB's existing late-binder will be updated to support these scenarios as well, rather than introduce a new one. However this isn't copy/paste for sure, but a more deeply applied unification of dynamic constructs within otherwise static code.

James

Posted on 12/28/2008 @ 12:30 PM
I have used Python (dynamically typed) with Delphi (statically typed) before. The Variant COM data type made it very convenient to do that.
I have not used C# lately and I am uncertain how the dynamic keyword adds further benefit than this datatype (or how it fits with C# now), but I welcome any support towards integrating dynamism.
Those of you who have used dynamically typed languages *in conjunction* with your statically typed languages don't know what you are missing.
There are often segments of code that are better done in a dynamic language and static languages NEED a clean way to access them.
Static analyzers like FxCop can quickly adapt and recognize these calls. So don't worry about it.

Jon Rista

Posted on 1/2/2009 @ 2:43 PM
I am so confused by everyone who get up in arms against new language features in C#. From the beginning, Microsoft has always said they want to make a language that empowers programmers to be PRODUCTIVE, not a language that is pure to any particular paradigm. It started out as an OO language, but it was never intended to stay that way. Just like C# programmers have learned to adore lambda expressions, anonymous types, auto-implemented properties, etc., so they will come to love dynamic as well. Like a few others have said...C# is a toolbox. A carpenter uses the right tool for the job..when a nail needs to be pounded into wood, a hammer is used, and when a screw needs to be tightened, a screwdriver is used. Its unlikely that a carpenter will use a screwdriver to hammer a nail.

Yes, there are bad programmers. Bad programmers tend use the wrong tool for the job, but thats the case regardless of what features a language may have. Its time to get over your insistance on language "purity", and embrace with Microsoft, .NET, and C# have to offer in terms of making YOU a more productive programmer. C# is becomming one of the richest and most advanced languages in history, and will change the face of how we develop software. Thats not going to change. If you prefer OO purity, use Java. If you prefer dynamic purity, use Python or Ruby. If you prefer functional purity, use F#. However, if you care more about developing a useful product that meets the demands of your customers in a timely manner...use C#. No other language will offer you the level of productivity of Microsofts flagship language...purity of productivity.
Post your comment and continue the discussion.