Improve Data Processing Performance of Dojo Grid

This article is reproduced from the open source Chinese community, the URL is http://www.oschina.net/question/12_8708

Author:< span class="dwauthor" style="padding:0px; margin:0px">Sun Yan, software engineer, IBM

Introduction: Dojo has introduced a powerful and robust control-Grid since 1.0. Programmers can use this control to make beautiful spreadsheets when developing Gui programs. The most important aspect of the Gui program is the user experience, but when adding a large amount of data to the Grid, the response of the program is usually very slow. This article uses some methods to improve the performance of Dojo Grid when adding data and enhance the user experience.

Dojo Grid is similar in structure to the familiar MVC pattern. The MVC pattern is the abbreviation of “Model-View-Controller”, that is, “model-view-controller”.

Figure 1. MVC structure

The structure of a simplest Grid mainly consists of the following aspects:

• Model

Each Grid will contain data, so the Model will be defined at the beginning of each Grid. As defined in Listing 1, Model contains dojotype (dojo model class), jsid (special id), structure (structure), Store (database), etc. The most important part is the Store, which places the data stored in the Grid.

List 1. Grid definition

 
 
 canSort="false" structure="modelGridLayout" Store="modelStore">
 

• View and Structure

View is used to define each data column, a View is a combination of multiple data columns. By defining the View, the Grid can display data as required. A structure is a collection of views, which means that multiple views can be combined into a structure. Structure will be used by Grid, and View will not be used directly by Grid, but will be packaged into a Structure for use. Listing 2 is an example of Grid Layout, which defines the structure of the Grid. The cells section defines the information defined by the Grid column. Each column needs to define the name, id, field, and the html format of the column such as length, width, and height. The subsequent operations on the Grid column are mainly for the field field.

List 2. Grid Layout definition

 
 
 ModelGridLayout = [{ 
 cells: [
 {name:'
 onclick="DeviceGridRevertSelectAll(this)" id="checkcollection">
', < br style="padding:0px; margin:0px"> field:'Sel', editable: true, width: '20px', cellStyles:'text-decoration: none; 
 cursor:default; text-align: center;position: relative; left: -10px', headerStyles: 
'text-align: center;', type: dojox .grid.cells.Bool }, 
 
 {name:'Model',field:'Model', width : '170px',cellStyles:'font-size:9pt; 
 cursor: default;text-align: left;', cellClasses:'defaultColumn', headerStyles: 
'text-align: center;'}, 
 
 {name:'Device',field:'Device', width: '150px', cellStyles:'font-size: 9pt; 
 font-style:normal,text-decoration: none; cursor:default;text-align: left;', 
 cellClasses:'defaultColumn', headerStyles:'text- align: center;'}, 
] 
 }]; 
 

• Grid Control (Widget)

The Grid control here is similar to the control in MVC器(Control). Through various predefined APIs of Grid, the data (Model) and view (View) of Grid are effectively organized and operated. In order to achieve the purpose of effectively controlling Grid data access, update, and appearance changes. This displays a Grid list similar to a spreadsheet.

Dojo Grid data storage

In the definition of Grid Model, there is an attribute called Store, which stores the data associated with the Grid , Which is the data source that the Grid is bound to. In the example, the name of the data source is modelStore. The definition of modelStore is as follows:

The core of Dojo provides a read-only data body implementation, ItemFileReadStore. This data body can read the Json structure from the HTTP endpoint, or read the JavaScript object in the memory. Dojo allows you to specify an attribute for ItemFileReadStore as an identifier (unique identifier). Dojo core also provides ItemFileWriteStore storage as an extension of ItemFileReadStore, which is built on the support of dojo.data.api.Write and dojo.data.api.Notification API for ItemFileReadStore. If your application needs to write data to ItemFileStore, ItemFileWriteStore is exactly what you want.

For Store operations, you can use the functions newItem, deleteItem, setValue to modify the contents of the Store. These operations can be called The revert function is used to cancel, or the save function is called to submit the modification.

When using these functions, it must be noted that if you specify a property for ItemFileWriteStore as the identifier , It must be ensured that its value is unique. This is particularly important for the newItem and deleteItem functions. ItemFileWriteStore uses these IDs to track these changes. This means that even if you delete an Item, its identity is still kept in use, so If you call newItem() and try to reuse this identifier, you will get an exception. To reuse this identifier, you need to commit your changes by calling save(). Save will apply all current changes and clear the pending state, including the reserved identifier. When you do not specify an identifier for the Store, the above problem will not occur. The reason is that the Store automatically creates an identifier for each Item and ensures that its value is unique. In this case of automatically creating an identifier, the identifier will not be exposed as a public attribute (it is also impossible to get it through getValue, only getIdentity can).

Store data is stored in two json arrays named _arrayOfAllItems and _arrayOfTopLevelItems. The difference between these two arrays is that the former records all the variables that have existed in the Store since the Grid was created, and the latter only stores all the current items in the Store. If a variable is deleted, the array variable in _arrayOfAllItems will be set to null, and the array variable in _arrayOfTopLevelItems will be completely deleted, and the number of arrays will be reduced by one. This setting is for Store.newItem, if the user does not specify an identifier for the Store, the Store can automatically use the number of _arrayOfAllItems to create an identifier for the new item. The number of _arrayOfAllItems will not be reduced by the delete operation, so there is no need to worry about the identifier of the new Item being repeated.

Listing 3 of the program describes the process of Grid automatically creating an identifier. First, the program will try to obtain the previously defined Identifier property. If the attribute is Number, assign _arrayOfAllItems.length to newIdentity. If the attribute is not Number, then the keywordArgs corresponding to the identifierAttribute is assigned to newIdentity. If newIdentity is empty, it means that the identity creation failed.

List 3. Grid identifier creation

 var newIdentity = null; 
 var identifierAttribute = this._getIdentifierA ttribute(); 
 if(identifierAttribute === Number){ 
 newIdentity = this._arrayOfAllItems.length; < br style="padding:0px; margin:0px">} 
 else{ 
 newIdentity = keywordArgs[identifierAttribute] ; 
 if (typeof newIdentity === "undefined"){ 
 throw new Error("newItem() was not passed an identity for the new Item"); 
} 
} 
 

Figure 2 shows the difference between _arrayOfAllItems and _arrayOfTopLevelItems after deleting a piece of data. A variable of _arrayOfAllItems is blanked, and the number of _arrayOfTopLevelItems is reduced by one.

Figure 2. arrayOfAllItems and arrayOfTopLevelItems
< /p>

Dojo Grid provides a large number of very useful APIs. Based on these functions, programmers can make beautiful spreadsheets. But in the course of use, some of its performance problems are gradually exposed. A prominent problem is that when using the newItem() method to add a large amount of data to the Grid, the browser will get busy due to the operation of the Grid, and will not respond for a long time or even a blank screen. For UI users, fast and stable page response is what they all expect. So is there any way to improve this problem?

Causes of performance problems

Figure 3. Grid modification data

The process of Grid modifying data is always to modify the data in the Store bound to the Grid first, and then change the View of the Grid as needed to complete the update of the external table of the Grid. Under normal circumstances, when Grid is created, if Store is defined, Grid will call this._setStore(this.Store); to configure its own Store properties. In the function _setStore, Grid listens to Store creation, deletion, and modification of Item events. Grid uses the dojo.connect() event model to bind a custom function to the Store. This function will be called whenever the Store calls onSet, onNew, and onDelete. This process is as described in Listing 4. Grid links the Store’s onSet, onNew, and onDelete events with its own defined _onSet._onNew,_onDelete through this.connect.

List 4. Grid connect event

 
 
 h.push(this.connect(this.Store, "onSet", "_onSet")); 
 h.push(this.connect(this .Store, "onNew", "_onNew")); 
 h.push(this.connect(this.Store, "onDelete", "_onDelete")); < br style="padding:0px; margin:0px"> 

Listing 5 shows the Store’s _onNew event handler. When the program uses newItem to add data to the Store, the Store will send out a dojo.data.api.Notification after completing the addition operation – this.onNew(newItem, pInfo); After the Grid listens to this Notification, it will call the previously defined The _onNew processing function to operate on itself. Grid will update its own number of rows, add new items (_addItem), and print some messages if necessary.

List 5. Store _onNew()

 
 
 _onNew: function(Item, parentInfo){ 
 this.updateRowCount(this.rowCount+1); // update the number of rows
 this._addItem(Item, this.rowCount-1); // add new Item 
 this.showMessage(); // print some information 
} 
 

< p style="padding-top:0px; padding-bottom:0px; margin-top:0px; margin-bottom:10px; font-family:'Microsoft YaHei',Verdana,sans-serif,SimSun; font-size:14px ; line-height:22px">

In the above three steps, adding a new Item is the most critical. The operation of _addItem includes obtaining the Identity of the new item, allocating the Identity space, filling the Item to the Identity space, and updating the row view (adding a new dom node). The specific process is shown in Listing 6.

List 6. Grid _addItem()

 
 
 _addItem: function(Item, index, noUpdate){ 
 var idty = this._hasIdentity? This.Store.getIdentity(Item): dojo.toJson(this. query) + 
 ":idx:" + index + ":sort:" + dojo.toJson(this.getSortProps()); // Get Identity 
 var o = {idty: idty, Item: Item }; // Allocate space
 this._by_idty[idty] = this ._by_idx[index] = o;// Fill the Identity space Item 
 if(!noUpdate){ 
 this .updateRow(index);// Update row view
} 
} 
 

Through the above analysis, we can see that if you use newItem adds data to the Store in a loop, then after the Store performs an add operation, it will cause the Grid to load the newly added item and update its view. After experimentation, it is found that the speed of adding items to the Store is very fast, while the operations of loading new items and updating its own view of the grid are relatively slow. Every time a new data is added, Grid will try to load the newly added data (create all dom nodes). Although this ensures that Grid gets all the current data in the Store, this operation increases the memory overhead of the browser. , And further make the browser busy and unresponsive for a long time.

Solutions to Grid performance problems

To solve the above performance problems, three problems need to be solved.

Step 1. Disconnect the Grid and Store in time

Because the speed of adding data to the Store is very fast, and the speed of loading new items and updating the view of the Grid is slow, so at the beginning of the operation, set the Store of the Grid to null. Open the connection between Grid and Store, deviceGrid._setStore(null);. In this way, even if Store changes due to addItem, it will not cause Grid operation in linkage. After adding data to the Store, link the Grid and Store: deviceGrid._setStore(deviceStore); Finally, reload the data to complete the view update: deviceGrid.render(). Listing 7 shows this process. At the beginning of adding data, modelGrid disconnects the Store first, then adds data to the modelStore, and then reconnects the modelStore to the modelGrid after the end.

List 7. Grid add items

 
 function _addGridData(dataarray) 
 {
 modelGrid._setStore(null); // Disconnect 
 /* Add data to the Grid*/ 
 for(var i=0;i var modelname = dataarray[i].split(',')[0] ; 
 var devicename = dataarray[i].split(',')[1]; 
 modelStore.newItem ({id:"modelItem"+i,StatusImage:'', 
 Sel:true, Loop:1,Status:'Not Started', 
 Model: modelname, Device: devicename} ); 
} 
 modelGrid._setStore(modelStore);// restore connection
 modelGrid.render();// Re-add it to the view
} 
 

Step2. 适应“lazy loading”数据加载机制

Dojo Grid 在获取Item 时有一种机制叫做“lazy loading”。在 Grid 初始化时并不把 Store 里的所有数据都加载进来， 而是采用“on-demand”的方式进行数据加载。触发数据加载的事件是 Grid 滚动条的的拖拉动作。当滚动条被拖拉到某一个特定位置时， Grid 会计算出当前滚动条的位置，并把和当前位置相关的数据加载进来。数据加载是按照“页”为单位装载的。有两个比较重要的属性： 
keepRows: 75 // Number of rows to keep in the rendering cache 
rowsPerPage: 25 //Number of rows to render at a time, and the rows number in each page 
在步骤 1 的最后，Grid 使用 render( ) 函数来取回表格、表头和视图，并把滚动条停留在 Grid 最顶端。因此，在 Grid 完成一次 render 后加载进来的数据只有 25 条。其余的数据要在用户拖动滚动条后触发再按需加载进来。

这样的数据加载方式，固然是减小了内存开销，提高了页面加载速度。但如果此时使用 Grid 的 getItem:(idx) 函数，程序会因为 getItem 函数中的 var data = this._by_idx[idx]; 语句而报错。因为输入函数 idx 可能大于当前 Grid 加载进来的数据量，此时通过 idx 去 Grid 中索引 Item 就会导致数组越界报错。也就是说在“lazy loading”的情况下，数据没有同时全部载入，这时如果企图通过 Grid.getItem(idx) 来操作 Store 中的所有数据的修改、删除是不安全的。

解决这一问题的方法是直接对 Store 中的数据进行获取、修改、删除。 Store 中的 _arrayOfTopLevelItems 存储着的 Store 当前数据。因为这些数据和 Grid 的显示 数据是一一对应，所以可以通过参数（Grid Item 的行数）把 Grid 的数据映射到 Store 中，直接对 Store 里的源数据进行操作。 

清单 8 中展示的是如何通过直接存取 Store 中的数据来实现对 Grid 数据的操作。程序的第一段定义了 GetItemfromStore 函数，他有两个参数一个是 Store 的名称和需要索引的行数。有了这两个参数，就可以通过 Store._arrayOfTopLevelItems[idx] 获得对应的存储在 Store 中 Item 了。 

修改 Item 比较简单，在函数 ModifyItem 中只要由 GetItemfromStore 得到 Item，然后对 Item 修改 setValue 就可以更改 Item 了。删除 Item 也同样是由 GetItemfromStore 得到 Item，然后取出 Item 中的某个属性判断该 Item 是否符合删除条件，如果符合就在 Store 中加以删除。结合 step1 中的操作 Store 前断开连接， 操作完 Store 后恢复连接，就可以实现 Grid 的删除功能。

清单 8. 对 Store 中存储的 Item 直接操作

				
 
 // 获得 Store 中的 Item 
 function GetItemfromStore(Store,idx) 
 { 
  var Item=eval(Store._arrayOfTopLevelItems[idx]); 
  return Item; 
 } 

 // 修改 Item 
 function  ModifyItem() 
 { 	
  for (var i=0;i<100;i++){ 
    var Item=GetItemfromStore(modelStore,i); 
    modelStore.setValue(Item,'Loop',i); 
    } 
 } 
 // 删除 Item 
 function DeleteItem() 
 { 
  var deletnu m=0; 
  var pushidx=new Array; 
  modelGrid._setStore(null);// 断开连接
  for(var i=0;i    var Item; 
    Item=GetItemfromStore(modelStore,i);// 获得 Item 
    if(Item !=null){ 
      var sel = modelStore.getValue(Item,'Sel');// 获得 Sel 属性   
      if(sel==true){ 
        deletnum=deletnum+1; 
        pushidx.push(Item);// 把符合条件的 Itempush 到 Array 中去
      } 
     }  
    } 
    var Items = pushidx; 
    /*Store 循环删除 Item*/ 
    if(Items.length){ 
    	 for(var i=0;i    		 modelStore.deleteItem(Items[i]); 
      } 
    } 
    modelGrid._setStore(modelStore);// 恢复连接
    modelGrid._refresh();//Grid 更新视图
 } 
     

Step3. 重构 Grid 的排序方法

Grid 具有排序功能，点击表头可以实现对表格内容升序或者降序排列。每次排序的操作都是针对 Grid 加载进来的数据进行的。排序后，当需要对数据操作时，还是先通过 Grid.getItem(idx) 获得 Item，然后依靠 Item 特有的 identifier 索引到 Store 中的真实数据， 再对数据进行修改。

然而因为“lazy loading”的存在，Grid 并没有把所有数据同时加载进来，这就导致了 Grid 排序后会出现数据获得错误和数据索引错误。所以为了保证数据索引正确，就需要从数据源上对数据进行排序，这样才能保证 Grid 和 Store 中的数据顺序保持一致。

具体做法是先禁用 Grid 的默认 sort 方法：canSort=”false”。然后重新定义 Grid.onHeaderCellMouseDown 的响应函数， 重构 sort 函数，以及更新 Grid 标题视图。这样就可以保证 Grid 的排序功能正常运行。

清单 9 展示了重新定义 onHeaderCellMouseDown 的响应函数的过程。函数定义了一个数组来存放临时变量，并且记录了表头上是否存着“全选”。接着函数寻找记录需要排序的项目，接着设置此次排序是顺序还是逆序排列。设置完成后，就调用自定义的 sort 函数对数据进行排序。排序完后 对表头进行一定的修改，增加一个向上或者向下的箭头来标识当前表格的某列是按升序还是降序排列，便于用户识别。

清单 9. 重新定义 onHeaderCellMouseDown 的响应函数

				
 //Grid.onHeaderCe llMouseDown 事件就是鼠标点击表头所触发的事件。 
 // 我们所要做的就是把这一事件的处理函数重定向到我们自己定义的排序方法。 
 modelGrid.onHeaderCellMouseDown = function(e){ 
 modelGrid._setStore(null); 
 var instancesArr = new Array(); // 定义一个数组存放排序临时数据
 var allselRrd=dojo.byId('checkcollection').checked;// 记录表头上的“全选”状态
 columnSort=e.cellIndex; 
 var propSort=modelGridLayout[0].cells[e.cellIndex].name;  // 记录排序的项目
 if(columnSort!=0){ 
  sortAscending=!sortAscending; // 设置正向排序还是逆向排序
  for(var i=0;i     instancesArr.push(modelStore._arrayOfTopLevelItems[i]); 
   } 
  sortmodelGrid(instancesArr,propSort); // 重写 sort 函数
  modelStore._arrayOfTopLevelItems=instancesArr; 
  modelGrid._setStore(modelStore); 
  UpdateHeaderView(); // 更新表头	
  } 
 dojo.byId('checkcollection').checked=allselRrd; 	
 }      

清单 10 是我们根据需要重写的排序函数。在这里主要是对数据做了一个分类处理。如果排序数据是数字的话，就按照大小排列。如果排序数据是子母的话， 就先把他们转换成小写子母，然后再根据子母顺序进行排序。

清单 10. 重构 Sort 函数

				
 // 根据所在列的内容的属性定制适合的 Sort 函数
 function sortmodelGrid(arr,propSorter) 
 { 
  var comp=1; 
  var asc=1; 
  if(sortAscending){ 
   asc=1;} 
  else{ 
   asc=-1;} 
  for(var i=0;i < (arr.length);i++){ 
   for(var j=0;j <(arr.length-1-i);j++){ 
    var aProp=eval("arr[j]."+propSorter+"[0]"); 
    var bProp = eval("arr[j+1]."+propSorter+"[0]"); 
    if(IsNumber(aProp)&& IsNumber(bProp)){ 
    // 如果是数字就直接排序
    } 
    else{ 
    // 如果是子母就先转换成小写再排序< br style="padding:0px; margin:0px">     aProp= aProp.toLowerCase(); 
     bProp = bProp.toLowerCase(); 
    } 
    if(aProp > bProp){ 
     comp=1;} 
    else if(aProp < bProp){ 
     comp=-1;} 
    else{ 
     comp=0;} 
    if((comp*asc) >0){ 
     var Itemm=arr[j+1]; 
     arr[j+1]=arr[j]; 
     arr[j]=Itemm; 	
    } 
  } 
 } 
 } 

清单 11 所做的就是在 Grid 的表头栏上增加一个向上或者向下的箭头来标识当前表格的某列是按升序还是降序排列，便于用户识别。操作的方法主要是通过根据一些 html 属性取出表头的各列的值，对其 html 语言进行修改，插入一个箭头的符号。

清单 11. 更新 Grid 表头视图

				
 // 增加一个向上或者向下的箭头来标识当前表格的某列是按升序还是降序排列，便于用户识别
 function UpdateHeaderView(){ 
 var docnObj=document.getElementsByTagName("th"); 
 for(var i=0;i < 5;i++){ 
  var docnObjName=modelGridLayout[0].cells[i].name; 
  var ret = [ '
  if(i==columnSort){ 
  // 通过判断 sortAscending 是 true 还是 false 来认知当前是升序还是降序排列
  // 根据排列顺序来修改表头的 css 
   ret = ret.concat([' ',(sortAscending ==true)?'dojoxGridSortUp':'dojoxGridSortDown','"> 
	 
',(sortAscending ==true)? '▲':'▼', 
	 '
' ]); 
   ret = ret.concat([propSort, '
']); 
  } 
  else{ 
   ret.push('">'); 
   ret = ret.concat([docnObjName, '']); 
  } 		
 docnObj[i].innerHTML=ret.join(" "); 
 } 
 } 

图 4 是 Grid 排序后的一张效果图。可以看到 Grid 中的各项已经在 Device 列上降序排列了。 Device 表头上多了一个向下的箭头，表示数据降序排列。

图 4.Grid 排序效果图

Grid 性能改善的前后对比

从下面的对比图中可以看到，经过改造后的 Grid，它在数据处理性能上的提高是非常巨大的。在 firefox 3.5 上测试，增加 142 个项目的时间由原来的接近 1 分钟的时间缩短到 1 秒以内。说明这种提高性能的方式是行之有效的。

图 5. 未进行性能优化耗费的时间图
 

图 6. 性能优化后耗费的时间图
 

图 7 优化后添加 / 删除耗费的时间图

小结

本文解释了 Dojo Grid 控件为何在数据处理上速度较慢的原因。并且在此基础上，提供了一种提高 Dojo Grid 处理数据速度的方法， 包括如何适时与 Store 断开或建立连接，如何为适应“lazy loading”特性更改获取数据方式以及如何重构排序函数。经过实验证明，该方法性能良好。

< CMA ID: 490811 > < Site ID: 10 > < XSLT stylesheet used to transform this file: dw-article-6.0-beta.xsl >

参考资料

学习

• “Dojo Wiki” 对 Dojo 的各种概念进行了全面的介绍，是了解基础概念的好地方。
  
  
• Dojo 官方网站提供了最全面的资料，包括 Dojo 源码，各种文档，实例等等。
  
  
• developerWorks Dojo 专题汇集了大量与 Dojo 相关的技术资源。
  
  
• developerWorks 技术活动和网络广播：随时关注 developerWorks 技术活动和网络广播。 
  
  
• developerWorks Web development 专区：通过专门关于 Web 技术的文章和教程，扩展您在网站开发方面的技能。
  
  
• developerWorks Ajax 资源中心：这是有关 Ajax 编程模型信息的一站式中心，包括很多文档、教程、论坛、blog、wiki 和新闻。任何 Ajax 的新信息都能在这里找到。
  
  
• developerWorks Web 2.0 资源中心，这是有关 Web 2.0 相关信息的一站式中心，包括大量 Web 2.0 技术文章、教程、下载和相关技术资源。您还可以通过 Web 2.0 新手入门 栏目，迅速了解 Web 2.0 的相关概念。