Frame
Extends:
This pure data structure represent a database like table internally it consists of rows and columns representing a generalized 2d matrix You can also think of this like a limited excel sheet. No operation will visibly modify the Frame, all operations create a new frame The class takes considerable effort to reuse as much of the internal data as possible to minimize the amount of garbage collection that is necessary.
It easily handles large number of rows and columns and has been tested to work well with over 1 million rows with hundreds of columns. The real limitation is the amount of ram available to Javascript.
Constructor Summary
Public Constructor | ||
public |
constructor(data: Array, columns: Array, name: [type], keyFunc: [type]) construct a new frame, Note once you have passed the data and column arguments, you no longer own those two items and should not modify them or any of their content |
Member Summary
Public Members | ||
public |
|
|
public get |
|
|
public get |
rows: [RowObject] Get an array of row objects |
|
public |
|
|
public get |
|
Private Members | ||
private |
|
|
private |
|
|
private |
|
|
private |
|
|
private |
|
Method Summary
Public Methods | ||
public |
asObj(ix: [type]): [type] [asObj description] |
|
public |
asObjList(): [RowObject] Return a list of RowObject from the frame |
|
public |
Concatinate frames and return a new frame (does not modify any of the input frames) |
|
public |
convertData(convList: null): Frame The default behavior of this method is to convert numeric strings into and return a new frame will the converted data. |
|
public |
distinct() |
|
public |
Takes a filter function or an array of row indicies and returns a new Frame with the selected rows |
|
public |
same as filter but returns the index of the filtered lines |
|
public |
find a row using the test function fn. |
|
public |
similar to an array forEach |
|
public |
groupBy(groupCols: [type], listOfAccumulatorFunctions: [type], toSort: [type]): [type] similar semantics to SQL groupBy, assumes list has bee sorted by 'groupCols' aFrame.groupBy(['mon',"weekday"], [gbSum('v1','sumV1'), gbCount('weekday', 'numWeekDays'), gbMin('v2', 'minV2'), gbMax('v1','maxV1'), gbMean('v1','meanV1') ]) |
|
public |
This is like SQL innerJoin join the rows where value of colFromThis == |
|
public |
leftJoin similar to sql left join, it is like innerJoin but all the rows of the 'this' are prensnt in the output |
|
public |
makeUnique(): [type] Remove all redundent copies of strings, this is the only destructive operation oan aframe while still maintaining functional purity |
|
public |
An alias for the Frame.mapF |
|
public |
Similar to Array.map |
|
public |
mergeCols(cols: array, someColsToMerge: array, newNameForMergerCol: String, sepOrMergeFunc: string | function): Frame Convert several columns into a single column, the elements rae converted to theeir string representation |
|
public |
Like leftJoin, but all th rows of both the left frame (this) and rightFrame ar present in the output |
|
public |
project(colsMapping: selected_columns, mappingObj: Object, filter: function, tester: function): Frame colsMapping can be a 1. |
|
public |
Similar to arry reduce except it works on frames |
|
public |
Rename columns, format for which [ 'oldColumnName=newColumnName' ...] |
|
public |
select a more convinient interface to project, an analog to sql select The primary diffenrence is the setting of new columns or adding new columns has been made more convinient |
|
public |
sort(colNames: [type], cmpFn: [type]): [type] [sort description] |
|
public |
toString() |
|
public |
Returns a function that can be used iteratively to project one array to another. |
|
public |
Convinience method as an alias for the project method |
|
public |
withIndex add an index column to the befining of the frame or at the end of the frame |
Private Methods | ||
private |
_genAuxJoinFilter(fn: Function, aFrame: [type]): [type] Creates a filter function the for the additional join checck Note the primary joining action (primary criterion) is the joinOp which look like this joinOp => 'commonColumnNameInBothFrames' or 'colFram1==colFrame2' |
|
private |
_rowObj This is an code efficient mathod of accessing the element of the 'rowLikeArray' using the columns names of the frame. |
|
private |
_toHtml(): [type] Return a string represent of the frame formatted as an HTML table |
Inherited Summary
From class BaseFrame | ||
public get |
columns: [type] [columns description] |
|
public get |
|
|
public get |
|
|
public get |
|
|
public |
|
|
public |
|
|
private |
|
|
private |
|
|
private |
|
|
private |
|
|
public |
asStrList(): [string] An arry of strings, where the string the is a list of tab seperated column values |
|
public |
colIx(name: [type]): [type] [colIx description] |
|
public |
column(colName: [type]): [type] Simple version of project to create a frame with only one column |
|
public |
equal() |
|
public |
[description] |
|
public |
forEachRaw(fn: function): [type] [description] |
|
public |
getKey(i: [type]): [type] [getKey description] |
|
public |
keyName() |
|
public |
This is the functional equivenent of Array.map for a Frame |
|
public |
Array of column names the are predominantly numeric, the values may be number or number like staring \ * @return {[string]} array of column names that are predominantly numeric |
|
public |
rawColumn get the column given by the 'colName' |
|
public |
Basically this is the equivelat to reduce performed on the data array, this is an array of rows as the elemnt passed to reduc |
|
public |
Get the row vector |
|
public |
|
|
public |
Create a new frame based of this frame but with new data |
|
public |
setKey() |
|
public |
setName() |
|
public |
Similar functionality to array slice |
|
private |
_getKey() |
Public Constructors
public constructor(data: Array, columns: Array, name: [type], keyFunc: [type]) source
construct a new frame, Note once you have passed the data and column arguments, you no longer own those two items and should not modify them or any of their content
Override:
BaseFrame#constructorParams:
Name | Type | Attribute | Description |
data | Array | the data fot the frame is an array of rows and each row is an array of strings or numbers |
|
columns | Array | array of strings representing the column names |
|
name | [type] | optional name for the frame |
|
keyFunc | [type] | optional function that is present will return the column represinting the unique key |
Public Members
public AccessClass source
public get numericColumns source
Array of column names the are predominantly numeric, the values may be number or number like staring \ * @return {[string]} array of column names that are predominantly numeric
Override:
BaseFrame#numericColumnspublic get rows: [RowObject] source
Get an array of row objects
Return:
[RowObject] | array of row objects |
public showLen source
public get unique source
Private Members
private _hash source
private _row source
private _rows source
private _sortCols source
private _unique source
Public Methods
public asObj(ix: [type]): [type] source
[asObj description]
Params:
Name | Type | Attribute | Description |
ix | [type] | [description] |
Return:
[type] | [description] |
public asObjList(): [RowObject] source
Return a list of RowObject from the frame
Return:
[RowObject] | [description] |
public concat(frames: [Frame]): Frame source
Concatinate frames and return a new frame (does not modify any of the input frames)
Params:
Name | Type | Attribute | Description |
frames | [Frame] | list of frames |
public convertData(convList: null): Frame source
The default behavior of this method is to convert numeric strings into and return a new frame will the converted data. It tries to reuse as much of the data as possible
If a conversion list is supplied, each element of the list consists of the following [ testFunc(aValue) -- This takes a value a returns true or false convertFn(aValue) -- and returns the new value ] for each cell we apply the testFunc if false then try the next testFunc in the list if the testFunc return true, the apply the convertFn to the cell value to the new value if non of the testFn succeeds then the cell remains unchanged
Params:
Name | Type | Attribute | Description |
convList | null | this parameter is optional |
public distinct() source
public filter(fnOrArray: function_array): Frame source
Takes a filter function or an array of row indicies and returns a new Frame with the selected rows
Params:
Name | Type | Attribute | Description |
fnOrArray | function_array | This is an array or a filter function (r:RowObject,ix,array) or an array of index into the data |
public filterIX(func: function): [int32] source
same as filter but returns the index of the filtered lines
Params:
Name | Type | Attribute | Description |
func | function | filter function |
Return:
[int32] | returns an array of row indexes (indicies) |
public find(fn: Function): RowObject source
find a row using the test function fn. The test function recieves a row ia s RowObject facade, see _rowObj method.
Params:
Name | Type | Attribute | Description |
fn | Function | function with a test criterion for the find operation |
Return:
RowObject | undefined if nothing found or a RowObject for the row matching the test function |
public forEachF(fn: function) source
similar to an array forEach
Params:
Name | Type | Attribute | Description |
fn | function | takes function(row:array, ix, array) |
public groupBy(groupCols: [type], listOfAccumulatorFunctions: [type], toSort: [type]): [type] source
similar semantics to SQL groupBy, assumes list has bee sorted by 'groupCols' aFrame.groupBy(['mon',"weekday"], [gbSum('v1','sumV1'), gbCount('weekday', 'numWeekDays'), gbMin('v2', 'minV2'), gbMax('v1','maxV1'), gbMean('v1','meanV1') ])
Params:
Name | Type | Attribute | Description |
groupCols | [type] | [description] |
|
listOfAccumulatorFunctions | [type] | [description] |
|
toSort | [type] | [description] |
Return:
[type] | [description] |
public innerJoin(rightFrame: Frame, colsToMap: Array, joinOn: string, filter: Function): Frame source
This is like SQL innerJoin join the rows where value of colFromThis ==
Params:
Name | Type | Attribute | Description |
rightFrame | Frame | the frame to join against |
|
colsToMap | Array | array of array of (strings) representing column names '1.colName' or '2.colName' |
|
joinOn | string | 'colFromThis==colFrom_aFrame' |
|
filter | Function | filter(rowObjectFronLeftFtame, rowObjectFromRightFlame) if true the join is possible |
public leftJoin(rightFrame: Frame, colsToMap: Array, joinOn: string, filter: function): Frame source
leftJoin similar to sql left join, it is like innerJoin but all the rows of the 'this' are prensnt in the output
public makeUnique(): [type] source
Remove all redundent copies of strings, this is the only destructive operation oan aframe while still maintaining functional purity
Return:
[type] | [description] |
public map(fn: function): Array source
An alias for the Frame.mapF
Params:
Name | Type | Attribute | Description |
fn | function | f(rowObject, ) |
public mapF(fn: function): Array source
Similar to Array.map
Params:
Name | Type | Attribute | Description |
fn | function | takes fn(rowObject, index, this.data ) |
public mergeCols(cols: array, someColsToMerge: array, newNameForMergerCol: String, sepOrMergeFunc: string | function): Frame source
Convert several columns into a single column, the elements rae converted to theeir string representation
e.g aFrame.columns = ['name','street','city','state','zip'] but we want to convert it into a ['name', 'address'] so: ['street','city','state','zip'] => ['address']
newFrame = aFrame.mergeCols(null, ['street','city','state','zip'], 'address', ", ");
public outerJoin(rightFrame: Frame, colsToMap: Array, joinOn: string, filter: function): Frame source
Like leftJoin, but all th rows of both the left frame (this) and rightFrame ar present in the output
public project(colsMapping: selected_columns, mappingObj: Object, filter: function, tester: function): Frame source
colsMapping can be a
1. rearranged list of col names from the frame e.g [ 'B', 'A', 'F']
2. or rearranged list of col names [
...('columnName' | 'oldName=newName' | 'newColumnName')
]
example of usage: list - Frame with list.data:
[ [ 'jan', 'mon', 1, 2 ],
[ 'jan', 'tue', 2, 6 ],
[ 'jan', 'wed', 3, 4 ],
[ 'jan', 'thu', undefined, 1 ],
...
list.columns: [ 'mon', 'weekday', 'v1', 'v2' ],
add column 'nurul' with value 123, change 'weekday' to uppercase, 'v1' conver undefined to 0
// Note the names if mapper are the new column names
var mapper = {
nurul: (v, rowObj, i,row) => "123",
weekday: (v) => v.toUpperCase(),
v1: (v) => v===undefined?0:v
};
list.project([ 'mon', 'weekday','nurul' , 'v1', 'v2' ], mapper);
result:
[ [ 'jan', 'MON', '123', 1, 2 ],
[ 'jan', 'TUE', '123', 2, 6 ],
[ 'jan', 'WED', '123', 3, 4 ],
'jan', 'THU', '123', 0, 1 ],
...
another expample:
// Note the use. of the now column names
function toNm(x) { x? (+x) : 0; }
var mapper1 = {
nurul: () => "123",
"Week Day": v => v?v.toUpperCase(): '',
v1: (v) => v===undefined?0:v,
v3: (v, rowObj) => toNm(rowObj.v1) + toNm(rowObj.v2) // sum of v1 and v2
};
list.project([ 'mon=Month', 'weekday=Week Day','nurul' , 'v1', 'v2', 'v3' ], mapper);
the result is the same as the previous example, but the columns names will now be ['Month', 'Week Day', 'nurul', 'v1', 'v2', 'v3' ]
[project description]
Params:
Name | Type | Attribute | Description |
colsMapping | selected_columns | array of strings |
|
mappingObj | Object | a mapping object with { newColumnNam: mappingFunction(oldCellValue, rowObj, rowNumber, rawRowArray)} |
|
filter | function | a filter function applied to the original row to determine if the row should be included |
|
tester | function | if a function to test if a column/row should have a mapping applied |
public reduce(fn: function, ini: any): any source
Similar to arry reduce except it works on frames
Params:
Name | Type | Attribute | Description |
fn | function | reduce function (acc:T, r:RowObject, index, array) |
|
ini | any | initial value of accumulator type T |
Return:
any | return the accumulator |
public rename(list: [string]): Frame source
Rename columns, format for which [ 'oldColumnName=newColumnName' ...]
Params:
Name | Type | Attribute | Description |
list | [string] | Array of strings |
Return:
Frame | A new Frame with the renamed columns, since these are immutable data structures all unmodified data are shared with the original frame |
public select(columns: [col_desciptor], where: function): Frame source
select a more convinient interface to project, an analog to sql select The primary diffenrence is the setting of new columns or adding new columns has been made more convinient
Example of setting columns [ 'oldColumn', 'oldColumn=newColumnName', ['oldCol1=newColName1', (v,ro) => some-operation], ['oldCol1=newColName1', someValue], ['newCol', someValue], ['newCol2', (v,ro) => someComputedValue] ]
Params:
Name | Type | Attribute | Description |
columns | [col_desciptor] | [description] |
|
where | function | this is a filter function similar to SQL select ... from table where expr |
public sort(colNames: [type], cmpFn: [type]): [type] source
[sort description]
Params:
Name | Type | Attribute | Description |
colNames | [type] | [description] |
|
cmpFn | [type] | [description] |
Return:
[type] | [description] |
public toString() source
public trProject(colsMapping: Array, mappingObj: Object): Function source
Returns a function that can be used iteratively to project one array to another. This is
Return:
Function | mapping function (inputArray, outputArray) rearranges the input array to the output array |
public update(mapper: Object, filter: function, tester: function): Frame source
Convinience method as an alias for the project method
Params:
Name | Type | Attribute | Description |
mapper | Object | Is an object with the keys being the column name to and the value is an update function or a constant |
|
filter | function | Optional filter function that will only keep rows if the filter function returns true, the filter operates on the original row not the result of the update |
|
tester | function | Optional tester function on the original row, if it returns true the update is performed other no update is done to the row |
public withIndex(indexName: boolean, atEnd: boolean): Frame source
withIndex add an index column to the befining of the frame or at the end of the frame
Private Methods
private _genAuxJoinFilter(fn: Function, aFrame: [type]): [type] source
Creates a filter function the for the additional join checck Note the primary joining action (primary criterion) is the joinOp which look like this joinOp => 'commonColumnNameInBothFrames' or 'colFram1==colFrame2'
Params:
Name | Type | Attribute | Description |
fn | Function | function to compare 2 rows a and b |
|
aFrame | [type] |
Return:
[type] |
private _rowObj(rowLikeArray: row, ix: int32): RowObject source
_rowObj This is an code efficient mathod of accessing the element of the 'rowLikeArray' using the columns names of the frame. The RowElement created is a facade to the rowLikeArray, it does not actuall copy the elements of the array to itself. It is just a convinient way of retrieving the elements of the array by name.
e.g. if the this.columns : ['name', 'age', 'sex']
let arr = ['Jane', 21, 'Female']; let ro = this._rowObj(arr,5);
ro.sex is the same as arr[2] == 'Female' ro.name === arr[0] ro._index$ === 5
Finally: arr[2] = 'Male'; ro.sex === 'Male'
Params:
Name | Type | Attribute | Description |
rowLikeArray | row | array representing the row, this array should have the same length as the rows in the frame |
|
ix | int32 | row number |
Return:
RowObject | row object representing the element |
private _toHtml(): [type] source
Return a string represent of the frame formatted as an HTML table
Return:
[type] | [description] |