Thoughts on the CSV File Format

As we all know, CSV files are used to manipulate data in many applications and databases.

With the arrival of the NoSQL-type files that use the JSON format and others for data storage and functionalities, it's still difficult to perform tasks for parser development. It consumes a lot of RAM. Since the year 2007, before the resurgence of NoSQL, I've been developing a different database type other than the RDBMS.

New Paradigm

In the year 2007, I started to develop a type of Database Manager System that was more flexible than the relational ones based on the CSV file format. I've already had some success with this. To do this, it was necessary to create a dedicated data management language and a parser for syntax evaluation, tools integration, and testing. Today, we can enjoy this work through CSV Comp DB (and its tools).

Safety

The CSV Comp DB has concurrency control with a global lock for each instance and can be used by multiple users concurrently.

Access

It can interact with an application with the use of a CQL (Comma Query Language) script. Such a script must be embedded in the programming language in which its application has been developed.

Example in Pascal:

uses system; //to execute CSV Parser.All OS
procedure Transaction;
var CQL:Tstringlist;
    DB,Table,code,name,email:string;
begin 
//aplly data to variables
code := '00001';
name := 'Almir_Bispo'; //Undescores cares
email := 'gibroh_webmaster@hotmail.com';
//Database
DB := 'c:\MyDB\';
Table: = 'Products';
//create a driver
CQL := Tstringlist.create;
//add CQL Command block to driver (Add data to Table)
CQL.add('{' + DB+Table+';@adicionar;(' + code+';'+name+';'+email + ');0;0;0;query=0;destino=0}');
//save the drive to CSV Comp DB  INPUT
CQL.savetofile(application.location+'inpout_comp.exe');
//Execute CSV Comp DB (Desktop Version)
system.ExecuteProcess('open',pchar(application.location+'CSV_PARSER.exe'),[]);
CQL.free;
Shomessage('Data as Sent !');
end;

//use on button click
Transaction;

We can create complex applications, including transactions. To help developers, a technology called ILDE was created.

About CQL

The CQL Script language follows rigid access rules for data table arrays (CSV type) and has several functions, including full editing functions, searches by text scan, by indexes, by fields, changes, and the creation of native web pages.

Note: CQL can be used both in data management and in the generation of web pages of a website.

CQL Paradigm

CQL is a typed, interpreted script system where you use parameters according to the syntax request. Its execution is defined by sequential blocks:

For example:

//Add data to "Table" on "DB"
{
..DB/Table;
    @adicionar;
    (id-code ; id-name ; id-email );
    0;
    0;
    0;
    query=0;
    destino=0
}
//The html page is created from "Table" and aplly css on server
{
..DB/Table;
    @html;
    1;
    4;
    Blue;
    (http://localhost/bealtyfull.css);
    query=0;
    destino=0
}

About ILDE

ILDE is a term to describe (intermediate level embedded Driver), which has CQL function blocks embedded.

We have open-source distributions for several languages:

• ILDE-Lua.

• ILDE-Pascal (Delphi, Lazarus).

• ILDE-Python.

• ILDE-Java.

• ILDE-Dart (new Google language).

Example ILDE-Lua:

function Transaction()
  local DB = "c:\\MyDB\\"
  local Table = "Products"
  local code = "00001"
  local Product = "Rice"
  local Price ="3,45" -- comma is decimal delimiter
  local fields = code..Products..Price  --concat all to fields
  CSV_COMP_DB_execute(  add_data(DB ,Table ,fields)  )--ILDE Lua is similar to java
end
--call
Transaction()

Where to Apply the CSV Comp DB

For the uses that I have already developed, it was possible to create several applications:

• Sites.

• E-commerce.

• Databases for web or desktop.

• Supervisors for sensors and microcontrollers.