Saturday, 18 May 2013 - Nesterovsky bros

Saturday, 18 May 2013

While trying to generalize our practices from KendoUI related projects we've participated so far, we updated control.js - a small javascript additions to KendoUI.

At present we have defined:

1. An extended model. See KendoUI extended model.

2. A lightweight user control - a widget to bind a template and a model, and to facilitate declarative instantiation. See KendoUI User control.

3. A reworked version of nesterovskyBros.defineControl() function.

var widgetType = scope.defineControl( { name: widget-name-string, model: widget-model-type, template: optional-content-template, windowOptions: optional-window-options }, base);

When optional-content-template is not specified then template is calculated as following:

var template = options.temlate || proto.template || model.temlate; if (template === undefined) { template = scope.template(options.name.toLowerCase() + "-template"); }

When windowOptions is specified then widgetType.dialog(options) function is defined. It's used to open dialog based on the specified user control. windowOptions is passed to kendo.ui.Window constructor. windowOptions.closeOnEscape indicates whether to close opened dialog on escape.

widgetType.dialog() returns a kendo.ui.Window instance with content based on the user control. Window instance contains functions:

result() - a $.Deffered for the dialog result, and
model() - referring to the user control model.

The model instance has functions:

dialog() referring to the dialog, and
result() referring to the dialog result.

widget.dialog() allows all css units in windowOptions.width and windowOptions.height parameters.

base - is optional user control base. It defaults to nesterovskyBros.ui.UserControl.

4. Adjusted splitter. See Adjust KendoUI Splitter.

5. Auto resize support.

Layout is often depends on available area. One example is Splitter widget that recalculates its panes when window or container Splitter is resized. There are other cases when you would like to adjust layout when a container's area is changed like: adjust grid, tab, editor or user's control contents.

KendoUI does not provide a solution for this problem, so we have defined our own.

A widget can be marked with class="auto-resize" marker;
A widget may define a widgetType.autoResize(element) function that adapts widget to a new size.
A code can call nesterovskyBros.resize(element) function at trigger resizing of the subtree.

To support existing controls we have defined autoResize() function for Grid, Splitter, TabStrip, and Editor widgets.

To see how auto resizing works, it's best to look into index.html, products.tmpl.html, and into the implementation controls.js.

Please note that we consider controls.js as an addition to KendoUI library. If in the future the library will integrate or implement similar features we will be happy to start using their API.

Extensions in `nesterovskyBros.data.Model`

As with kendo.data.Model there is fields Object - a set of key/value pairs to configure the model fields, but fields have some more options:
- fields.fieldName.serializable Boolean - indicates whether the field appears in an object returned in model.toJSON(). Default is true.
- fields.fieldName.updateDirty Boolean - indicates whether the change of the property should trigger dirty field change. Default is true.
When model defines a field and there is a prototype function with the same name then this function is used to get and set a field value.
When property is changed through the model.set() method then dirty change event is triggered (provided that fields.fieldName.updateDirty !== false). This helps to build a dependcy graph on that property.
When model instance is consturcted, the data passed in are validated, nullable and default values are set.

Model example

Here is an example of a model:

nesterovskyBros.data.ProductModel = nesterovskyBros.data.Model.define( { fields: { name: { type: "string", defaultValue: "Product Name" }, price: { type: "number", defaultValue: 10 }, unitsInStockValue: { type: "number", defaultValue: 10, serializable: false }, unitsInStock: { type: "string" } }, unitsInStock: function(value) { if (value === undefined) { var count = this.get("unitsInStockValue"); return ["one", "two", "three", "four"][count] || (count + ""); } else { this.set("unitsInStockValue", ({one: 1, two: 2, three: 3, four: 4 })[value] || value); } } });

Notice that:

unitsInStock property is implemented as a function - this helps to map model values to presentation values.
when you call model.toJSON(), or JSON.stringify() you will see in result name, price, unitsInStock values only - this helps to get model's state and to store it somewhere (e.g. in sessionStorage).
in a code:
var model = new nesterovskyBros.data.ProductModel({ price: "7", unitsInStock: "one" });
the following is true:
(typeof(model.price) == "number") && (mode.price == 7) && (model.name == "Product Name") && (model.unitsInStockValue == 1)

As with UserControl the implemntation is defined in the controls.js. The sample page is the same index.html

Wednesday, 03 April 2013 20:37:49 UTC

Comments [0] -
javascript | Thinking aloud | Tips and tricks

Thursday, 28 March 2013

Windows 8 + Lenovo Yoga-13 + external SD card

Two weeks ago we've gotten new Lenovo 13" laptops (Yoga-13 with touch screens and Windows 8 Pro on board).

The first expression was WOW! Touch screens! Windows 8! Now we'll try our hand on that new (for us) API. So new, so cool...

A day later. What a shit this new UI. Where are my desktop, "Start" button, all the programs... After googling we've understood - we're not alone.

Few more days later. We've recognized that our SSD hard disk won't live long life with our projects. We generates output several GB a day. Thus we've decided to buy external SD cards - additional 64Gb, class 10. That's enough for us. No sooner said than done. After several attempts to copy our projects from hard drive to SD card (~9Gb of sources) we strongly believe that such a vigorous mix (Lenovo + Win 8 + external SD card) won't survive. Windows 8 hangs up when display off (in middle of data copy, after an hour of work). What a .... of .... this Windows 8, Lenovo and SD cards all together.

Thursday, 28 March 2013 22:39:55 UTC

Comments [2] -
Thinking aloud

Tuesday, 26 March 2013

KendoUI User control

Developing with KendoUI we try to formalize tasks. With this in mind we would like to have user controls.

We define user control as following:

It is a javascript class that extends Widget.
It offers a way to reuse UI.
It allows to define a model and a template with UI and data binding.

Unfortunately, KendoUI does not have such API, though one can easily define it; so we have defined our version.

Here we review our solution. We have taken a grid KendoUI example and converted it into a user control.

User control on the page

See index.html

<!DOCTYPE html> <html> <head> <title>Test</title>  <script src="scripts/templates.js"></script> <script src="scripts/jquery/jquery.js"></script> <script src="scripts/kendo/kendo.web.min.js"></script>  <script src="scripts/controls.js"></script>  <script src="scripts/controls/confirm.js"></script>  <script src="scripts/controls/products.js"></script> <link href="styles/kendo.common.min.css" rel="stylesheet" /> <link href="styles/kendo.default.min.css" rel="stylesheet" /> <script> $(function () { // (5) Bind the page. kendo.bind( document.body, // (6) Model as a datasource. { source: [new nesterovskyBros.data.ProductsModel] }); }); </script> </head> <body>  <div data-role="products" data-bind="source: source"></div> </body> </html>

That's what we see here:

Templates that define layouts. See "How To: Load KendoUI Templates from External Files", and templates.tt.
Definition of the UserControl widget.
Confirm dialog user control (we shall mention it later).
Products user control.
Data binding that instantiates page controls.
Model is passed to a user control through the dataSource.
Use of Products user control. Notice that "data-role" defines control type, "source" refers to the model.

User Control declaration

Declaration consists of a view and a model.

View is html with data binding. See products.tmpl.html

We build our project using Visual Studio, so templates packaging is done with templates.tt. This transformation converts products template into a tag:

<script id="products-template" type="text/x-kendo-template">

thus template can be referred by a utility function: nesterovskyBros.template("products-template").

Model inherits kedo.data.Model. Here how it looks:

// (1) Define a ProducsModel class. nesterovskyBros.data.ProductsModel = kendo.data.Model.define( { // (2) Model properties. fields: { productName: { type: "string", defaultValue: "Product Name" }, productPrice: { type: "number", defaultValue: 10 }, productUnitsInStock: { type: "number", defaultValue: 10 }, products: { type: "default", defaultValue: [] } }, // (3) Model methods. addProduct: function () { ... }, deleteProduct: function (e) { ... }, ... }); // (4) Register user control. nesterovskyBros.ui.Products = nesterovskyBros.defineControl( { name: "Products", model: nesterovskyBros.data.ProductsModel });

That's what we have here:

We define a model that inherits KendoUI Model.
We define model fields.
We define model methods.
Register user control with nesterovskyBros.defineControl(proto) call, where:
- proto.name - defines user control name;
- proto.model - defines model type;
- proto.template - defines optional template. If not specified, a template is retrieved from $("#" + proto.name.toLowerCase() + "-template").html().

UserControl API

Now, what's remained is API for the UserControl. See controls.js.

UserControl defines following events:
- change - triggered when data source is changed;
- dataBound - triggered when widget is data bound;
- dataBinding - triggered befor widget data binding;
- save - used to notify user to save model state.
UserControl defines following options:
- autoBind (default false) - autoBind data source;
- template (default $.noop) - user control template.
UserControl defines dataSource field and setDataSource() method.
UserControl defines rebind() method to manually rebuild widget's view from the template and model.
UserControl sets/deletes model.owner, which is a function returning a user control widget when model is bound/unbound to the widget.
When UserControl binds/unbinds model a model.refresh method is called, if any.
You usually define you control with a call nesterovskyBros.defineControl(proto). See above.
There is also a convenience method to build a dialog based on a user control: nesterovskyBros.defineDialog(options), where
- options.name - a user control name (used in the data-role);
- options.model - a model type;
- options.windowOptions - a window options.
This method returns a function that recieves a user control model, and returns a dialog (kendo.ui.Window) based on the user control.
Dialog has model() function that returns an instance of model.
Model has dialog() function that returns an instance of the dialog.
Dialog and model have result() function that returns an instance of deferred object used to track dialog completion.
The example of user control dialog is confirm.js and confirm.tmpl.html. The use is in the products.js deleteProduct():

deleteProduct: function(e) { var that = this; return nesterovskyBros.dialog.confirm( { title: "Please confirm", message: "Do you want to delete the record?", confirm: "Yes", cancel: "No" }). open(). center(). result(). then( function(confirmed) { if (!confirmed) { return; } ... }); }

Last

User controls along with technique to manage and cache templates allow us to build robust web applications. As the added value it's became a trivial task to build SPA.

See also: Compile KendoUI templates.

Sunday, 06 January 2013 19:43:19 UTC

Comments [0] -
.NET | ASP.NET | javascript | Tips and tricks

Tuesday, 20 November 2012

StreamResource

Our goal is to generate reports in streaming mode.

At some point we need to deal with data streams (e.g. xml streams for xslt transformations). Often a nature of report demands several passes through the data. To increase performance we have defined a class named StreamResource. This class encapsulates input data, reads it once and caches it into a temp file; thus data can be traversed many times. StreamResource can read data lazily or in a eager way thus releasing resources early. This class can be used as a variation of PipeStream, which never blocks, as if a size of a buffer is not limited, and which can be read many times.

The API looks like this:

public class StreamResource: IDisposable { /// <summary> /// Creates a StreamSource instance. /// </summary> /// <param name="source"> /// A function that returns source as an input stream. /// </param> /// <param name="settings">Optional settings.</param> public StreamResource(Func<Stream> source, Settings settings = null); /// <summary> /// Creates a StreamSource instance. /// </summary> /// <param name="source"> /// A function that writes source data into an output stream. /// </param> /// <param name="settings">Optional settings.</param> public StreamResource(Action<Stream> source, Settings settings = null); /// <summary> /// Gets an input stream. /// </summary> /// <param name="shared"> /// Indicates that this StreamResouce should be disposed when returned /// stream is closed and there are no more currently opened cache streams. /// </param> /// <returns>A input stream.</returns> public Stream GetStream(bool shared = false); }

The use pattern is following:

// Acquire resource. using(var resource = new StreamResource(() => CallService(params...))) { // Read stream. using(var stream = resource.GetStream()) { ... } ... // Read stream again. using(var stream = resource.GetStream()) { ... } }

StreamResource is efficient even if you need to process content only once, as it monitors timings of reading of source data and compares it with timings of data consumption. If the difference exceeds some threshold then StreamResource caches source greedily, otherwise source is pooled lazily. Thus, input resources can be released promptly. This is important, for example, when the source depends on a database connection.

The use pattern is following:

// Acquire resource and get shared stream. using(var stream = new StreamResource(() => CallService(params...)).GetStream(true)) { ... }

Finally, StreamResource allows to process data in a pipe stream mode. This is when you have a generator function Action<Stream> that can write to a stream, and you want to read that data. The advantage of StreamResource over real pipe stream is that it can work without blocking of generator, thus releasing resources early.

The use pattern is similar to the previous one:

using(var stream = new StreamResource(output => Generate(output, params...)).GetStream(true)) { ... }

The source of the class can be found at Streaming.zip.

Tuesday, 20 November 2012 07:01:57 UTC

Comments [0] -
.NET | Thinking aloud

Friday, 09 November 2012

Change column type in SQL Server, Part 2

Two monthes ago we have started a process of changing column type from smallint to int in a big database.

This was splitted in two phases:

Change tables and internal stored procedures and functions.
Change interface API and update all clients.

The first part took almost two monthes to complete. Please read earlier post about the technique we have selected for the implementation. In total we have transferred about 15 billion rows. During this time database was online.

The second part was short but the problem was that we did not control all clients, so could not arbitrary change types of parameters and of result columns.

All our clients use Entity Framework 4 to access the database. All access is done though stored procedures. So suppose there was a procedure:

create procedure Data.GetReports(@type smallint) as begin select Type, ... from Data.Report where Type = @type; end;

where column "Type" was of type smallint. Now we were going to change it to:

create procedure Data.GetReports(@type int) as begin select Type, ... from Data.Report where Type = @type; end;

where "Type" column became of type int.

Our tests have shown that EF bears with change of types of input parameters, but throws exceptions when column type has been changed, even when a value fits the range. The reason is that EF uses method SqlDataReader.GetInt16 to access the column value. This method has a remark: "No conversions are performed; therefore, the data retrieved must already be a 16-bit signed integer."

Fortunately, we have found that EF allows additional columns in the result set. This helped us to formulate the solution. We have updated the procedure definition like this:

create procedure Data.GetReports(@type int) as begin select cast(Type as smallint) Type, -- deprecated Type TypeEx, ... from Data.Report where Type = @type; end;

This way:

result column "Type" is declared as deprecated;
old clients still work;
all clients should be updated to use "TypeEx" column;
after all clients will be updated we shall remove "Type" column from the result set.

So there is a clear migration process.

P.S. we don't understand why SqlDataReader doesn't support value conversion.

Friday, 09 November 2012 16:41:27 UTC

Comments [0] -
.NET | SQL Server puzzle | Tips and tricks

Monday, 29 October 2012

Export data to Excel

If you deal with web applications you probably have already dealt with export data to Excel. There are several options to prepare data for Excel:

generate CSV;
generate HTML that excel understands;
generate XML in Spreadsheet 2003 format;
generate data using Open XML SDK or some other 3rd party libraries;
generate data in XLSX format, according to Open XML specification.

You may find a good article with pros and cons of each solution here. We, in our turn, would like to share our experience in this field. Let's start from requirements:

Often we have to export huge data-sets.
We should be able to format, parametrize and to apply different styles to the exported data.
There are cases when exported data may contain more than one table per sheet or even more than one sheet.
Some exported data have to be illustrated with charts.

All these requirements led us to a solution based on XSLT processing of streamed data. The advantage of this solution is that the result is immediately forwarded to a client as fast as XSLT starts to generate output. Such approach is much productive than generating of XLSX using of Open XML SDK or any other third party library, since it avoids keeping a huge data-sets in memory on the server side.

Another advantage - is simple maintenance, as we achieve clear separation of data and presentation layers. On each request to change formatting or apply another style to a cell you just have to modify xslt file(s) that generate variable parts of XLSX.

As result, our clients get XLSX files according with Open XML specifications. The details of implementations of our solution see in our next posts.

Monday, 29 October 2012 15:34:38 UTC

Comments [0] -
.NET | ASP.NET | Thinking aloud | xslt

Over optimization in sql server

Recently we had a discussion with DBA regarding optimization strategey we have selected for some queries.

We have a table in our database. These are facts about that table:

the table is partitioned by date;
each partition contains a month worth of data;
the table contains at present about 110 million rows;
the table ever grows;
the table is most accessed in the database;
the most accessed part of the data is related to last 2-3 days, which is about 150000 rows.

The way we have optimized access to that table was a core of the dispute.

We have created filtered index that includes data for the last 3 days.

To achieve desired effect we had to:

create a job that recreates that index once a day, as filter condition is moving;
adjust queries that access the table, as we had to use several access pathes to the table depending on date.

As result we can see that under the load, stored procedures that access that table became almost 50% faster. On the other hand maintainance became more complicated.

DBA who didn't like the database complications had to agree that there are speed improvements. He said that there should be a better way to achieve the same effect but could not find it.

Are there a better way to optimize access to this table?

Monday, 29 October 2012 08:34:39 UTC

Comments [0] -
SQL Server puzzle | Tips and tricks

Navigation