Tuesday, 30 July 2013 - Nesterovsky bros

What is the slowest function in kendo? Or better, which function has most negative performance impact in kendo.

Recently, we were dealing with a simple page, which was too slow, as data binding took more than second.

The page contained a dropdown list, with ~1000 options. To understand the reason we have run this page under the IE's built-in javascript profiler, and ...

there, we have found that almost half of the time is taken by a function (call it X), which receives nothing and returns always the same result!

But, let's now see a minimal example that demostrates the problem:

<!DOCTYPE html> <html> <head> <title>Test</title> <script src="scripts/jquery/jquery.js"></script> <script src="scripts/kendo/kendo.web.min.js"></script> <link href="styles/kendo.common.min.css" rel="stylesheet" /> <link href="styles/kendo.default.min.css" rel="stylesheet" /> <link href="styles/style.css" rel="stylesheet" /> <script> var model; function init() { var source = []; for(var i = 0; i < 1000; ++i) { source.push({ text: "value " + i, value: "" + i }); } model = kendo.observable( { value: "1", source: new kendo.data.DataSource( { data: source }) }); model.source.read(); } function test() { kendo.bind("#view", model); } init(); </script> </head> <body> <p> <button onclick="test()">Click to start test</button> </p> <p id="view"> Select: <input data-role="dropdownlist" data-bind="value: value, source: source" data-text-field="text" data-value-field="value"/> </p> </body> </html>

There are two parts: initialization, and a test itself that starts upon button click.

In the initialization part we have defined a model, containing a datasource.

The test part performs data binding.

Now, here is a run statistics:

Function	Count	Inclusive time (ms)	Inclusive time %	Avg time (ms)
test	1	456.05	100	456.05
X	1,000	200.14	43.89	0.2

So, X is fast by itself, but it run 1000 times, and took about 44% of all time.

And now, to the function. It's kendo.data.binders.widget.source.fn._ns.

Here is its code:

_ns: function(ns) { ns = ns || kendo.ui; var all = [ kendo.ui, kendo.dataviz.ui, kendo.mobile.ui ]; all.splice($.inArray(ns, all), 1); all.unshift(ns); return kendo.rolesFromNamespaces(all); }

We can see that on each call it receives a parameter undefined, and always returns an array with the same content. Not sure why Kendo UI team made it so complex, but one can easily device a simple patch that optimizes this code path.

function patch() { var base = kendo.data.binders.widget.source.fn._ns; var result; kendo.data.binders.widget.source.fn._ns = function(ns) { return ns ? base.call(this, ns) : (result || (result = base.call(this, ns))); } }

With this patch applied, the profiler shows:

Function	Count	Inclusive time (ms)	Inclusive time %	Avg time (ms)
test	1	253.03	100	253.03
_ns	1,000	6	2.37	0.01

Execution time dropped considerably, and _ns() loses its title of most time consuming function!

An example can be found at slow.html.

Tuesday, 30 July 2013 20:47:11 UTC

Comments [0] -
javascript | kendoui

Visual to logical conversion in Java

Time after time we run into the same problem on different platforms, with different languages. The problem's name is "Visual to Logical conversion for right-to-left or bidirectional text". The problem is usually due to legacy code, which stores texts in visual order from left to right. In case of English it's ok, but with Hebrew this means that texts are partially reversed.

It worth to note that we've solved the same task with Windows API for native and .NET applications more than 10 years ago.

On the other hand, for Java, we yet didn't see any acceptable standalone solution. To remedy this omission, we publish here our solution to this problem.

package com.nesterovskyBros.text;

import java.text.Bidi;

/**
 * Utility that uses {@link Bidi} class. 
 */
public class BidiUtils
{
  /**
   * Implements visual to logical order converter.
   * 
   * @author <a href="http://www.nesterovsky-bros.com">Nesterovsky bros</a>
   *
   * @param text an input text in visual order to convert.
   * @return a String value in logical order.
   */
  public static String visualToLogical(String text)
  {
    if ((text == null) || (text.length() == 0))
    {
        return text;
    }
  
    Bidi bidi = new Bidi(text, Bidi.DIRECTION_DEFAULT_LEFT_TO_RIGHT);
  
    if (bidi.isLeftToRight())
    {
        return text;
    }
  
    int count = bidi.getRunCount();
    byte[] levels = new byte[count];
    Integer[] runs = new Integer[count];
  
    for (int i = 0; i < count; i++)
    {
       levels[i] = (byte)bidi.getRunLevel(i);
       runs[i] = i;
    }
  
    Bidi.reorderVisually(levels, 0, runs, 0, count);

    StringBuilder result = new StringBuilder();

    for (int i = 0; i < count; i++)
    {
       int index = runs[i];
       int start = bidi.getRunStart(index);
       int end = bidi.getRunLimit(index);
       int level = levels[index];
  
       if ((level & 1) != 0)
       {
         for (; --end >= start;)
         {
           result.append(text.charAt(end));
         }
       }
       else
       {
         result.append(text, start, end);
       }
    }

    return result.toString();
  }
}

This method utilizes embeded Bidi's algorithm, see class java.text.Bidi.

Be aware that there is no perfect algorithm that covers all possible cases, since BIDI was written for an opposite task, but our implementation based on Bidi.reorderVisually is usually acceptable.

Here is an JUnit test for this method:

package com.nesterovskyBors.text;

import static org.junit.Assert.*;

import org.junit.Test;

import com.nesterovskyBros.text.BidiUtils;

public class BidiUtilsTests
{

  @Test
  public void testsVisualToLogical()
  {
    String text = "123 יתימאה ןחבמ";
    String actual = BidiUtils.visualToLogical(text);
    String expected = "מבחן האמיתי 123";
  
    assertEquals(expected, actual);
    
    text = "תירבע English תירבע בוש";
    actual = BidiUtils.visualToLogical(text);
    expected = "שוב עברית English עברית";
    
    assertEquals(expected, actual);
  }
}

Sunday, 28 July 2013 15:44:17 UTC

Comments [0] -
Java | Tips and tricks

On Garbage Collectors

Recently we've seen an article Why mobile web apps are slow.

While title mentions web apps, the the criticism is directed purely to javascript language. The answer presented there is twofold:

Raw javascript performance is ~5 times slower than performance of native code solving the same task.
This accounts for the most modern implementations that exploit JIT. Author does not expect that this proportion will be significatly changed in javascript's favor in the near future.
Garbage Collection, which is essential part of javascript, does not work well in constrainted environment of mobile devices.
Here author quotes many references that show that:
- for GC to work on peer with non-GC application, it needs to have ~6 - 8 times size of memory than an application needs;
- at the same time for hardware reasons, mobile devices cannot provide such amount of memory;
- on the other hand with rise of CPU performance, GC pressure rises even faster.

In the end author, while saying about some attempts to change the state, has no final verdict, whether there can be anything done to remedy the problem.

Having roots in C++, we're GC unbelievers. But you know, who will ask your opinion on that, while there are all those modern languages that try to abstract from memory and implicitly or explicitly assume GC: java, C#, javascript, xslt, xquery, and so on.

There always is a solution to avoid GC completely, like C++ and other (e.g. Microsoft's C++/CX, or Apple's ARC) do. But, assuming you're entered GC world, what can be done with it? How do you make it more predictable, less greedy, and probably more fast?

Our arguments are like this.

How does native code manage object graphs?

Today's solution is reference counting along with weak references to break cycles in graph.

Can be GC based on this?

Yes.

In fact upcoming spec of javascript contains weak references. So, provided a developer accurately defines relations in an object graph, one may immediately achieve the same efficiency as native solution.

If one does not use weak references consistently then object cycles can be created, so there can appear a graph that is not referenced anywhere in a program. This graph can be collected with classical GC that scans object roots.

Classical GC part can be used as a debug mode leak detector, and will collect graph cycles at runtime.

Thus, we claim that a hybrid memory management: reference counting with weak references plus classical GC is possible; it will be equal to a native memory management when weak references are properly used, and it will be close to classical GC without use of weak references.

This solution gives a rule to mitigate GC drawbacks: just use weak references in appropriate points, and you can continue to live in GC world, where GC is only a fallback.

Sunday, 14 July 2013 12:20:29 UTC

Comments [0] -
javascript | Thinking aloud

Prises to RESTful

RESTful is well known and established archetecture. It does not need our approvals or recomendations.

It's something different that we want to express. We'd like to thank all those people who formulated those ideas.

While one can argue that REST has "always" existed along with the web, from our experience we can see that most of web applications we have created up to the late 200x were stateful.

We can see that many web applications should support the application (session) state. And here REST has given a rule:

client stores a session state;
server does not store a session, and is represented as set of services;
if there is a state that cannot be stored on a client (e.g. due to security reasons), then server should use caches, and be able to reconstruct that state upon cache miss.

Now is a good time for the proliferation of the REST, as even weakest clients (browsers) can implement its requirements.

REST allowed us to drastically simplify development and support, to unload server, and to build better web applications.

We compare web applications we have written a decade ago using ASP.NET, and in last 3-4 years using RESTful ideas both in java and in .NET. Clearly, the later have better performance, but from the support standpoint the most appealing is that you can instantly upgrade the application without impacting users, as there are no sessions on the server. For the same reason you should not puzzle over whether you should use in-process sessions and session stickness with load balancing server, or out-of-process sessions.

Things became simpler:

server now is pure logic through services (WCF, Web API, JAX-RS);
client is gui - jquery, kendoui or other;
aspx/jsf pages gone completely;

Friday, 12 July 2013 13:25:28 UTC

Comments [0] -
Thinking aloud

Compile KendoUI templates

Earlier, in the article How To: Load KendoUI Templates from External Files, we were talking about the way to combine project's templates into a single file using Text Templates. Now, we would like to suggest the next step.

KendoUI defines text templates that it knows to transform into functions, at runtime obviously. Thus a template like this:

<tr> <td data-bind=" text: name"></td> <td>#: kendo.toString(get("price"), "C") #</td> <td data-bind="text: unitsInStock"></td> <td><button class="k-button" data-bind="click: deleteProduct"> Delete</button></td> </tr>

is transformed into a function:

function(data) { var o,e=kendo.htmlEncode; with(data) { o='<tr><td data-bind="text: name"></td><td>'+ e( kendo.toString(get("price"), "C") )+ '</td><td data-bind="text: unitsInStock"></td>' + '<td><button class="k-button" ' + 'data-bind="click: deleteProduct">Delete</button></td></tr>'; } return o; }

The transformation is done through a sequence of of regex replaces.

Now, what's the fastest javascript template engine?

Right! That, which does not work at runtime. :-)

What we thought is that we can generate those functions at compile time rather than defining templates.

We have updated templates.tt to generate template functions, and optionally to generate <script> tags that call those functions. This way, for an input footer.tmpl.html:

<!DOCTYPE html> <html> <head> <title>Test</title> <base href="/" /> <link href="styles/kendo.common.min.css" rel="stylesheet" /> <link href="styles/kendo.default.min.css" rel="stylesheet" /> </head> <body> <table data-template-id="view"> <tr> <td>Products count: #: total() #</td> <td>Total price: #: totalPrice() #</td> <td colspan="2">Units in stock: #: totalUnitsInStock() #</td> </tr> </table> </body> </html>

templates.js will look like this:

nesterovskyBros.templates= { ... "footer-view":function(data) { var o,e=kendo.htmlEncode; with(data) { ... } return o; }, ... }; document.write('<script id="footer-view-template" type="text/x-kendo-template">#=nesterovskyBros.templates["footer-view"](data)#</script>');

To get template function at runtime you simply refer to nesterovskyBros.templates["footer-view"].

template.tt now allows you to specify:

scope - a javascript scope for tempate functions, e.g. "nesterovskyBros.templates";
data-script attribute over each template (default is true) to prevent generation of <script> tag;
data-with-block attribute (default is true) to prevent with(data) {...} statement in javascript.

See a sample application that shows how nicely KendoUI UserControls work with those compiled templates.

Sunday, 07 July 2013 18:54:57 UTC

Comments [2] -
javascript | kendoui | Thinking aloud

Derived xml schemas

Awhile ago we have created a set of xml schemas and xslt to represent different languages as xml, and to generate source from those xmls. This way we know to represent and generate: java, c#, cobol, and several sql dialects (read about languages xom on this site).

Here, we'd like to expose a nuisance we had with sql dialects schema.

Our goal was to define a basic sql schema, and dialect extensions. This way we assumed to express general and dialect specific constructs. So, lets consider an example.

General:

-- Select one row select * from A

DB2:

select * from A fetch first row only

T-SQL:

select top 1 * from A

Oracle:

select * from A where rownum = 1

All these queries have common core syntax, while at the same time have dialect specific means to express intention to return first row only.

Down to the xml schema basic select statement looks like this:

<xs:complexType name="select-statement"> <xs:complexContent> <xs:extension base="full-select-statement"> <xs:sequence> <xs:element name="columns" type="columns-clause"> <xs:element name="from" type="from-clause" minOccurs="0"> <xs:element name="where" type="unary-expression" minOccurs="0"/> <xs:element name="group-by" type="expression-list" minOccurs="0"/> <xs:element name="having" type="unary-expression" minOccurs="0"/> <xs:element name="order-by" type="order-by-clause" minOccurs="0"/> </xs:sequence> <xs:attribute name="specification" type="query-specification" use="optional" default="all"/> </xs:extension> </xs:complexContent> </xs:complexType>

Here all is relatively clear. The generic select looks like:

<sql:select> <sql:columns> <sql:column wildcard="true"/> </sql:columns> <sql:from> <sql:table name="A"/> </sql:from> </sql:select>

But how would you define dialect specifics?

E.g. for T-SQL we would like to see a markup:

<sql:select> <tsql:top> <sql:number value="1"/> </tsql:top> <sql:columns> <sql:column wildcard="true"/> </sql:columns> <sql:from> <sql:table name="A"/> </sql:from> </sql:select>

While for DB2 there should be:

<sql:select> <sql:columns> <sql:column wildcard="true"/> </sql:columns> <sql:from> <sql:table name="A"/> </sql:from> <db2:fetch-first rows="1"/> </sql:select>

So, again the quesions are:

how to define basic sql schema with goal to extend it in direction of DB2 or T-SQL?
how to define an xslt sql serializer that will be also extendable?

Though we have tried several solutions to that problem, none is satisfactory enough. To allow extensions we have defined that all elements in sql schema are based on sql-element, which allows extensions:

<xs:complexType name="sql-element" abstract="true"> <xs:sequence> <xs:element ref="extension" minOccurs="0" maxOccurs="unbounded"/> </xs:sequence> </xs:complexType> <xs:element name="extension" type="extension"/> <xs:complexType name="extension" abstract="true"> <xs:complexContent> <xs:extension base="sql-element"/> </xs:complexContent> </xs:complexType> ... <xs:element name="top" type="top-extension" substitutionGroup="sql:extension"/> <xs:complexType name="top-extension"> <xs:complexContent> <xs:extension base="sql:extension"> <xs:sequence> <xs:element ref="sql:expression"/> </xs:sequence> <xs:attribute name="percent" type="xs:boolean" use="optional" default="false"/> </xs:extension> </xs:complexContent> </xs:complexType>

Unfortunately, this creates too weak typed schema for extensions, thus intellisence suggests too many options.

Wednesday, 03 July 2013 05:50:43 UTC

Comments [0] -
Thinking aloud | xslt

A bit of science fiction?

On a way to a home we usually listen audio books in our car. In most cases these are science fiction stories. One of our favorite authors is Ray Bradbury. Probably you know his short story collection "The Martian Chronicles". Also, we like to glance over technology and science news to stay tuned in the latest innovations.

Recently, we've read an article that states that NASA is going to send a group of scientists-colonists to Mars in an observable future. It sounds in Ray Bradbury style.

Although, we're not specialists in this field, we discussed what difficulties may face such an expedition.

As far as we understand, there are following issues that must be solved before expedition may start:

speed of a spaceship is yet too slow
a shield from radiation doesn't exist
big cargo section to contain sufficient reserve of water, food and oxygen for the expedition to survive
a way to produce enough energy to survive on Mars

Actually, the second issue depends on the first one. At present there is no reliable long term protection from cosmic radiation, which can be installed on a spacecraft. At least we didn't hear about such thing. Thus, long staying in the open space will seriously harm colonists and may bring to naught the whole mission.

Сonsidering these facts, we concluded that a space travel further than the moon is not possible at present.

What could be done to solve these issues?

As a solution of space colonization could be the following. The earthlings won't send people but robots with corresponding equipment and containers for man, animals and plants DNAs or embryos. At the destination place robots will build a colony and will begin to grow people, animals, plants; to train them and to serve them later (at least to man <img alt= " src="http://www.nesterovsky-bros.com/weblog/smilies/wink.gif"/> ).

How all this may solve the mentioned issues?

It's much easier to create reliable shield from radiation for small DNA or embrios containers.
time and speed of a spacecraft, in such case, will impact less on the mission.
weight of spacecraft in this case could be less, or it may get more payload.

Thus, to bring the era of cosmic expansion, human kind must invest in the development of robotics, artificial intelligence, genetic engineering, development of rapid learning, among other scientific fields in space exploration.

Right now, you may see a beginning? of this trend in sciense here DFKI's robot ape to colonize the Moon?

In the far future, after the beginning will be forgotten, all this may lead to question: who was the first a man or a robot? <img alt= " src="http://www.nesterovsky-bros.com/weblog/smilies/wink.gif"/>

Wednesday, 26 June 2013 20:23:35 UTC

Comments [0] -
Thinking aloud

Kendo progress utility

While developing with KendoUI we have found kendo.ui.progress(container, toggle) function to be very useful. It's used to show or hide a progress indicator in the container element.

At the same time we have found that we usually used it in a context of async operation. This way, we want to show progress, perform some asynchronous operations, hide progress. So, we clearly want to benifit from RAII pattern: we would like to open a progress scope, and to perform some activity withing this scope.

Arguing like this, we have defined a utility function, which is the fusion of kendo.ui.progress() and $.when(). Its signature is like this:

nesterovskyBros.progress = function(instance /*, task ... */)

where instance is either Model, Widget, JQuery or DOM Element, and task is one or more deferred objects. This function shows a progress and returns a Promise that will hide a progress when all tasks will be complete. Implementation is trivial, so we quote it here:

// Fusion of kendo.ui.progress() and $.when(). scope.progress = function(instance /*, task ... */) { if (instance instanceof Model) { instance = instance.owner && instance.owner(); } if (instance instanceof Widget) { instance = instance.element; } if (instance && instance.nodeType) { instance = $(instance); } var id = ns + "-progress"; // "nesterovskyBros-progress"; var progress = (instance && instance.data(id)) || 0; if (arguments.length < 2) { return progress; } var result = $.when.apply(null, [].slice.call(arguments, 1)); if (instance) { instance.data(id, ++progress); kendo.ui.progress(instance, progress > 0); result.always( function() { progress = instance.data(id) || 0; instance.data(id, --progress); kendo.ui.progress(instance, progress > 0); }); } return result; };

The use is like this:

nesterovskyBros.progress(element, $.ajax("/service1"), $.ajax("/service2")).then(myFunc);

The code can be found at controls.js.

Tuesday, 28 May 2013 05:54:52 UTC

Comments [0] -
javascript | kendoui | Tips and tricks

KendoUI controls.js

While trying to generalize our practices from KendoUI related projects we've participated so far, we updated control.js - a small javascript additions to KendoUI.

At present we have defined:

1. An extended model. See KendoUI extended model.

2. A lightweight user control - a widget to bind a template and a model, and to facilitate declarative instantiation. See KendoUI User control.

3. A reworked version of nesterovskyBros.defineControl() function.

var widgetType = scope.defineControl( { name: widget-name-string, model: widget-model-type, template: optional-content-template, windowOptions: optional-window-options }, base);

When optional-content-template is not specified then template is calculated as following:

var template = options.temlate || proto.template || model.temlate; if (template === undefined) { template = scope.template(options.name.toLowerCase() + "-template"); }

When windowOptions is specified then widgetType.dialog(options) function is defined. It's used to open dialog based on the specified user control. windowOptions is passed to kendo.ui.Window constructor. windowOptions.closeOnEscape indicates whether to close opened dialog on escape.

widgetType.dialog() returns a kendo.ui.Window instance with content based on the user control. Window instance contains functions:

result() - a $.Deffered for the dialog result, and
model() - referring to the user control model.

The model instance has functions:

dialog() referring to the dialog, and
result() referring to the dialog result.

widget.dialog() allows all css units in windowOptions.width and windowOptions.height parameters.

base - is optional user control base. It defaults to nesterovskyBros.ui.UserControl.

4. Adjusted splitter. See Adjust KendoUI Splitter.

5. Auto resize support.

Layout is often depends on available area. One example is Splitter widget that recalculates its panes when window or container Splitter is resized. There are other cases when you would like to adjust layout when a container's area is changed like: adjust grid, tab, editor or user's control contents.

KendoUI does not provide a solution for this problem, so we have defined our own.

A widget can be marked with class="auto-resize" marker;
A widget may define a widgetType.autoResize(element) function that adapts widget to a new size.
A code can call nesterovskyBros.resize(element) function at trigger resizing of the subtree.

To support existing controls we have defined autoResize() function for Grid, Splitter, TabStrip, and Editor widgets.

To see how auto resizing works, it's best to look into index.html, products.tmpl.html, and into the implementation controls.js.

Please note that we consider controls.js as an addition to KendoUI library. If in the future the library will integrate or implement similar features we will be happy to start using their API.

Extensions in `nesterovskyBros.data.Model`

As with kendo.data.Model there is fields Object - a set of key/value pairs to configure the model fields, but fields have some more options:
- fields.fieldName.serializable Boolean - indicates whether the field appears in an object returned in model.toJSON(). Default is true.
- fields.fieldName.updateDirty Boolean - indicates whether the change of the property should trigger dirty field change. Default is true.
When model defines a field and there is a prototype function with the same name then this function is used to get and set a field value.
When property is changed through the model.set() method then dirty change event is triggered (provided that fields.fieldName.updateDirty !== false). This helps to build a dependcy graph on that property.
When model instance is consturcted, the data passed in are validated, nullable and default values are set.

Model example

Here is an example of a model:

nesterovskyBros.data.ProductModel = nesterovskyBros.data.Model.define( { fields: { name: { type: "string", defaultValue: "Product Name" }, price: { type: "number", defaultValue: 10 }, unitsInStockValue: { type: "number", defaultValue: 10, serializable: false }, unitsInStock: { type: "string" } }, unitsInStock: function(value) { if (value === undefined) { var count = this.get("unitsInStockValue"); return ["one", "two", "three", "four"][count] || (count + ""); } else { this.set("unitsInStockValue", ({one: 1, two: 2, three: 3, four: 4 })[value] || value); } } });

Notice that:

unitsInStock property is implemented as a function - this helps to map model values to presentation values.
when you call model.toJSON(), or JSON.stringify() you will see in result name, price, unitsInStock values only - this helps to get model's state and to store it somewhere (e.g. in sessionStorage).
in a code:
var model = new nesterovskyBros.data.ProductModel({ price: "7", unitsInStock: "one" });
the following is true:
(typeof(model.price) == "number") && (mode.price == 7) && (model.name == "Product Name") && (model.unitsInStockValue == 1)

As with UserControl the implemntation is defined in the controls.js. The sample page is the same index.html

Wednesday, 03 April 2013 20:37:49 UTC

Comments [0] -
javascript | Thinking aloud | Tips and tricks

Extensions in nesterovskyBros.data.Model

Model example

Extensions in `nesterovskyBros.data.Model`