Functions without dependencies

cMQL provide 3 aways to write Javascript in cMQL like way

  • js functions
  • cMQL wrappers
  • Clojure like language(Wisp)

When we need to combine some javascript calls we use cMQL wrappers.
When we need more advanced features like loops,map,filter etc we use wisp.

Function dependencies

No matter which way it will be used cMQL allows functions to have functions dependencies. We can't use dependencies,but cMQL provides a way to write functions and if we say what other functions they need,cMQL will auto-combine them.

For example if f3 depends on f1 f2,f2 depends on f1,and f1 doesn't have any dependency.

We make 5 files (inside js/lib)

  • f1.js (with the code)
  • f2.js
    f2.deps [f1] (will contain 1 vector only with the dependencies)
  • f3.js
    f3.deps [f2 f1] (order is not important,cMQL will find that we need f1 first)

cMQL will produce this standalone code js/f3.js (f1 code is only in f3 body not in f2 also)

function f3()
{
var f1=function f1(){...}
var f2=function f2(){...calls-f1...}
...f3-body...
}

This is done automatically,the user only gives the body of a function and 1 vector with its dependencies(if it has any).

Directory structure

For now a standard folder structure is needed.
We place it in the project folder,or the folder that the standalone.jar is

  • ๐Ÿ“ js
    • ๐Ÿ“ core
    • ๐Ÿ“ library (optional libraries)
    • ...stand-alone-functions....

Important Run the code with leinengen,if you use Intellijea go to run options.
Else the working directory will be different and cMQL will not be able to find the files.

js/core

for internal use only,it contains the wisp core language.

js/library

we put there our javascript functions 1 function/js file or wisp functions 1 function/wisp file

Our function f1 for example should have

  • f1.js
  • f1.deps optional (a vector with the names of the other function it needs)

Even if the function has no dependencies(its standalone),still put it in library, so some other function can use it.

Before using a function that exists in the library do

(compile-functions :f1 :f2 ...)

And f1.js f2.js will appear as standalone function in the js/

js/

For auto-generated standalone functions.
We can put js functions there also,but dont do it,unless the function is standalone, and you dont want to use it as a library to other function.

Use JS

How to use JS from cMQL.

  • Create for example 1 js function and save it in js/lib/assoc_js.js
    if it has dependencies add also a js/lib/assoc_js.deps (with the dependencies vector)

  • Compile it
    Always put your code in js/lib and compile before use
    (compile :assoc_js)
    After complile 1 standalone functions will be created js/assoc_js.js

  • Use it
    (ejs :assoc_js [arg1 arg2 ...])

    ejs is like the $function aggregate operator

cMQL wrappers

This is the first and simple way to write cMQL like code,even if we call javascript.

How it works

  • create small javascript functions
  • wrap them with cMQL
  • nest them to make more complicated code

We still write small javascript functions,but the code looks like cMQL code

cMQL optimizes the nested calls and allow them to be fast
For example even if we nest 10 $function calls,cMQL will generate only 1 to run on the server From benchmarks that can help like 3x compared to sending the 10 nested function to server directly

How to use

  • Create for example 2 js functions and save them in js/lib

    js/lib/assocjs.js

    function assoc_js(o, k, v)
    {
    o[k]=v;
    return o;
    }

    js/lib/dissocjs.js

    function dissoc_js(obj,k)
    {
    delete obj[k];
    return obj;
    }
  • Create 2 wrappers for them (optional)

    (defn assocjs [m k v]
    (njs :assocjs [m k v]))
    (defn dissocjs [m k]
    (njs :dissocjs [m k]))
  • Execute them,nest them in any way,and run them,for example

    (ejs (dissocjs (assocjs (assocjs :mydoc 'e' 5) 'f' 6) 'f'))

    The last ejs will make 1 function from the 3 nested ones.
    For MongoDB will be 1 $function operator with 1 js function making it fast.
    We can nest ejs also,without the use of njs,but when tested it was 2x slower.

  • If you dont want to create the wrappers you can still use njs for example But its hard even for 3 levels,and doesnt look Clojure like function calls

    (ejs (njs :dissocjs
    [(njs :assocjs
    [(njs :assocjs
    [:mydoc "e" 5])
    "f" 6]])
    "f"]))
  • How it works internally cMQL will find the f1,f2 js code,and will produce 1 function

    (f1 (f2 arg1 arg2) arg3)
    //will generate
    function (arg1,arg2,arg3)
    {
    f2=code_of_f2;
    //and then call the nested function
    return f1(f2(arg1,arg2),arg3);
    }

    This can be done for many functions,up to many nested levels not only 2

Clojurescript like language

The above method works if we want to combine some functions,but if we need more complicated things like loops etc,this approach is better.

cMQL uses compiles wisp to js(using wisp compiler)

Install wisp,so cMQL can use it internally

  • npm install wisp -g

How to use

  1. Write the wisp function

    Create a file js/lib/f1.wisp for example (write Clojurescript like wisp code)
    If it has dependencies create also f1.deps file with the dependencies vector like before.

    (compile-functions :f1)

    js/lib/f1.js will appear (not standalone) js/f1.js will appear (standalone)

    cMQL will run the js/f1.js , when i use it for example in (ejs :f1 ...)
    No need to do anything else

Perfomance

Results are based on very simple benchmark just to get a general idea.
They are not to be trusted,but it seems that both the cMQL wrappers and wisp are usable.

  1. Cost of Javascript
    Javascript is at least 2x slower than native aggregate operators
    Javascript is supposed to be used only if aggregation framework cannot be used
  2. Cost of wrappers
    1x-1.3x
    1.3x is for 10 levels of nesting
  3. Cost of wisp
    generated code ~1.2x slower from hand-written
    because we need the core also,we go to 2x-3x slower
    (2x is only for the "optimized core",that looks more like javascript)

Summary

  • Use MQL unless no solution or no fast solution in MQL
  • Use wrappers and nest code to make more complex code
  • If you need more advanced features like loops use wisp or javascript