Client-side search webpart extension for filtering on followed sites

As all of you know SharePoint 2013 comes with a much improved search webpart infrastructure. One can even build search filters based on user profile properties in order to build personalized experiences (refer to the Technet article Query variables in SharePoint 2013).
Strangely, it is currently not possible to filter search results based on the followed sites. Take the scenario where you have hundreds of sites that have stored documents, list items, etc. and you want to show a list of recently added items to the user for all his followed sites.

(people that can't wait and want the ready-to-use solution: go straight to the ce_followedsites.html gist. Others, read along!)

JavaScript-only solutions have their limitations
Most examples in the blogosphere provide a JavaScript focused solution where you need to include the specific business rules into code for filtering and rendering. What I started to wondering whether it would be possible to extend the search webparts with a custom token expression. The whole benefit would be that the filtering business rules can be expressed by the power user using the query builder, and the rendering logic with a custom display template.

Cloud-ready solutions are preferred
Although a full-trust solution based on inheriting the OOTB webparts should be doable, most customers don't want to invest anymore in this route, but want to focus on cloud-ready customizations, i.e. it should be applicable to SharePoint Online. So I started investigating.

The idea is to catch the search queries before they are about to be sent to the server, and then inject the queries with the user's followed sites data. Therefore it is important to configure your search webparts so that the search queries are executed client-side. You can find this option in the options tab within the query builder.

Microsoft AJAX
First direction I took was to find out the whold wiring that takes place when the page is loaded. In the server-side generated aspx that is pushed to the browser I see the following statements appearing:

Sys.Application.add_init(function() {
    $create(Srch.ContentBySearch, {"alternateErrorMessage":"","delayLoadTemplateScripts":true,"groupTemplateId":"~sitecollection/_catalogs/masterpage/Display Templates/Content Web Parts/Group_Content.js","itemTemplateId":"~sitecollection/_catalogs/masterpage/Display Templates/Content Web Parts/Item_mydisplaytemplate.js","messages":[],"numberOfItems":50,"queryGroupName":"Default","renderTemplateId":"~sitecollection/_catalogs/masterpage/Display Templates/Content Web Parts/Control_mydisplaytemplate.js","shouldHideControlWhenEmpty":false,"showBestBets":false,"showDataErrors":true,"showDefinitions":false,"showDidYouMean":false,"showPersonalFavorites":false,"states":{}}, null, null, $get("ctl00_ctl54_g_96bccb53_bb46_4003_bbba_9765befa5ebf_csr"));
});
Sys.Application.add_init(function() {
    $create(Srch.DataProvider, {"availableSorts":[],"bypassResultTypes":true,"clientType":"ContentSearchRegular","collapseSpecification":"","delayLoadTemplateScripts":true,"enableInterleaving":false,"fallbackSort":[],"hitHighlightedProperties":["Title","Path","Author","SectionNames","SiteDescription"],"initialQueryState":{"k":"","o":null,"s":0,"r":null,"l":0,"m":"","d":0,"x":null,"e":-1},"maxPagesAfterCurrent":1,"messages":[],"processBestBets":false,"processPersonalFavorites":false,"properties":{"TryCache":true,"Scope":"{Site.URL}","UpdateLinksForCatalogItems":true,"EnableStacking":true,"ListId":"a42ba038-b804-4126-afe7-467143dd9777","ListItemId":41},"queryGroupName":"Default","queryPropertiesTemplateUrl":"querygroup://webroot/Paginas/Test-SearchFollowedSites.aspx?groupname=Default","queryTemplate":"{FollowedSites} (contentclass:STS_Site OR contentclass:STS_Web)","rankRules":[],"renderTemplateId":"DefaultDataProvider","resultsPerPage":50,"selectedProperties":["Path","Title","FileExtension","SecondaryFileExtension"],"selectedRefiners":[],"sourceID":"8413cd39-2156-4e00-b54d-11efd9abdb89","sourceLevel":"Ssa","sourceName":"Local SharePoint Results","states":{},"trimDuplicates":false}, null, null, $get("ctl00_ctl54_g_96bccb53_bb46_4003_bbba_9765befa5ebf_ctl00_csr"));
});

So what is this Sys-namespace and its Sys.Application? It appears to the Microsoft Ajax framework that is leveraged here.


The add-init-function adds callbacks onto a list of functions that will be called when the html document is ready, by the function Sys.Application.Init as part of the Microsoft AJAX framework.
Would it be that easy to intercept this add_init-event, scan the applied queries, replace the tokens and fire the original $create-function calls?

Unfortunate initialization wiring
Unfortunately, it isn't, as I recognized that there is also the Sys.Component flow that will initialize all registered components during start-up of the page. So we can't execute our components later in the lifecycle easily. After some digging through the JavaScript-file search.ClientControls.debug.js I found the function Srch.ScriptApplicationManager.prototype.$4b_1 which is responsible for kicking off the queries and wiring the results. But as you know, you should only integrate with APIs found in JavaScript files that start with sp and secondly, these kinds of obscure function names should be avoided at all times. Ok, so I looked for another route.

Another direction: intercept SearchExecutor
I went looking for the async-ajax-calls that are eventually put on the wire in order to request for search results. I then could intercept these async calls and inspect the queries inside in order to replace the token with the followed sites. Luckily there was this clear async call indeed: a SearchExecutor-request.

So I started with a simple interception:

var oldexecuteQueries = Microsoft.SharePoint.Client.Search.Query.SearchExecutor.prototype.executeQueries;
Microsoft.SharePoint.Client.Search.Query.SearchExecutor.prototype.executeQueries = 
   function() {  
      return oldexecuteQueries.apply(this, arguments); 
   }

So far so good. And by the way, this function is part of the file sp.search.debug.js so one is allowed to integrate with this API. Nevertheless, as soon as I introduced a second async call, things went ugly:

var oldexecuteQueries = Microsoft.SharePoint.Client.Search.Query.SearchExecutor.prototype.executeQueries; 
Microsoft.SharePoint.Client.Search.Query.SearchExecutor.prototype.executeQueries = function() {  
   var myargs = Array.prototype.slice.call(arguments);
   var oldargs = fetchFollowedSites(function() {
      oldexecuteQueries.apply(that, myargs);  
      ?? how to provide the result to the original caller?
   }
   return;
}

Also intercept executeQueryAsync
Nice we are able to replace the query with the followed sites, but how can we return the new result to the original caller, i.e. the search webpart? The trick is to also intercept the original executeQueryAsync and wait with executing it until we have fetched the followed sites!

var oldexecuteQueryAsync = SP.ClientRuntimeContext.prototype.executeQueryAsync;
SP.ClientRuntimeContext.prototype.executeQueryAsync = function() {
   if (somePendingQuery) {
      ...
      jQuery.when.apply(jQuery, this.pendingQueries).then(function() {
         ...
  oldexecuteQueryAsync.apply(that, myargs);
      });
   }
   else return oldexecuteQueryAsync.apply(this, arguments);
}

(left out some fragments for brevity; please look up full gist via the link shared above) 

And there you have it. A client-side solution that just intercepts search queries as they are put on the wire and replaces tokens with the user's followed sites. Enjoy!

Full solution available
Full solution available via the following gist: ce_followedsites.html