Modules
- kstore-core : the core engine
- kstore-hdfs : HDFS support additional module
- kstore-s3 : S3 support additional module
- kstore-sample : samples using KStore
Main abstractions
KStore
Abstraction of a table, a KStore owns a schema defined as a list of columns, and stores data in buckets. A Device is associated to the KStore to define the physical storage.
Bucket
Defines an abstract storage of data rows.
PageBucket
Implementation of a Bucket.
Column & ColumnType
Abstraction of a column, mainly define by a type.
Line
A vector allowing reading one line of data when reading buckets.
Device
The physical abstraction of a storage, which may be file system, hadoop, s3, azure, whatever…
Configuration
Property file configuration.
Default implementations
DefaultKStore
A default implementation of a KStore.
DefaultColumn
A default implementation of a Column.
DefaultLine
A default implementation of a Line.
FileSystemDevice
A device implementing regular file systems.
HDFSDevice (in kstore-hdfs)
A device implementing HADOOP file system.
S3Device (in kstore-s3)
A device implementing S3 file system.
A 5 minute start guide
// Create a new schema with columns and types
List<Column> schema = Lists.newArrayList(new DefaultColumn(ColumnType.STRING));
// Build a new kstore in a folder. Right now, we're using a temporary directory
Path temp = Files.createTempDirectory("kstore");
KStore kStore = new DefaultKStore("My KStore", schema, temp.toString());
// Create a new bucket to store / retrive our data, and
// add two lines
Bucket bucket = kStore.newBucket();
bucket
.add(0, new Object[]{"France"})
.add(1, new Object[]{"Spain"})
.commit();
// Prepare to traverse our bucket. We want all lines from 0 to 2
Range range = new Range(0, 2);
// When traversing, we want to retrieve the column 0
Line line = new DefaultLine(0);
// Iterate through the bucket and print each line
bucket.readLines(line, range, (int rowId, Line l) -> showMeTheLine(rowId, l));
// Free resources
kStore.close();
Other operations
Saving / Loading
bucket.save (); // save the store metadata
bucket.load (); // load the store metadata
Deleting lines
bucket.deleteRowId(2); // delete the line with identifier 2
Rollbacking
bucket.rollback(); // cancel any operation since the last commit
