Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Thoughts on the CSV File Format

DZone's Guide to

Thoughts on the CSV File Format

Flexible Database Manager Systems are important for effectively manipulating databases. Learn how I developed a flexible system based on the CSV file format.

· Database Zone
Free Resource

Whether you work in SQL Server Management Studio or Visual Studio, Redgate tools integrate with your existing infrastructure, enabling you to align DevOps for your applications with DevOps for your SQL Server databases. Discover true Database DevOps, brought to you in partnership with Redgate.

As we all know, CSV files are used to manipulate data in many applications and databases.

With the arrival of the NoSQL-type files that use the JSON format and others for data storage and functionalities, it's still difficult to perform tasks for parser development. It consumes a lot of RAM. Since the year 2007, before the resurgence of NoSQL, I've been developing a different database type other than the RDBMS.

New Paradigm

In the year 2007, I started to develop a type of Database Manager System that was more flexible than the relational ones based on the CSV file format. I've already had some success with this. To do this, it was necessary to create a dedicated data management language and a parser for syntax evaluation, tools integration, and testing. Today, we can enjoy this work through CSV Comp DB (and its tools).

Safety

The CSV Comp DB has concurrency control with a global lock for each instance and can be used by multiple users concurrently.

Access

It can interact with an application with the use of a CQL (Comma Query Language) script. Such a script must be embedded in the programming language in which its application has been developed.

Example in Pascal:

uses system; //to execute CSV Parser.All OS
procedure Transaction;
var CQL:Tstringlist;
    DB,Table,code,name,email:string;
begin 
//aplly data to variables
code := '00001';
name := 'Almir_Bispo'; //Undescores cares
email := 'gibroh_webmaster@hotmail.com';
//Database
DB := 'c:\MyDB\';
Table: = 'Products';
//create a driver
CQL := Tstringlist.create;
//add CQL Command block to driver (Add data to Table)
CQL.add('{' + DB+Table+';@adicionar;(' + code+';'+name+';'+email + ');0;0;0;query=0;destino=0}');
//save the drive to CSV Comp DB  INPUT
CQL.savetofile(application.location+'inpout_comp.exe');
//Execute CSV Comp DB (Desktop Version)
system.ExecuteProcess('open',pchar(application.location+'CSV_PARSER.exe'),[]);
CQL.free;
Shomessage('Data as Sent !');
end;

//use on button click
Transaction;

We can create complex applications, including transactions. To help developers, a technology called ILDE was created.

About CQL

The CQL Script language follows rigid access rules for data table arrays (CSV type) and has several functions, including full editing functions, searches by text scan, by indexes, by fields, changes, and the creation of native web pages.

Note: CQL can be used both in data management and in the generation of web pages of a website.

CQL Paradigm

CQL is a typed, interpreted script system where you use parameters according to the syntax request. Its execution is defined by sequential blocks:

For example:

//Add data to "Table" on "DB"
{
..DB/Table;
    @adicionar;
    (id-code ; id-name ; id-email );
    0;
    0;
    0;
    query=0;
    destino=0
}
//The html page is created from "Table" and aplly css on server
{
..DB/Table;
    @html;
    1;
    4;
    Blue;
    (http://localhost/bealtyfull.css);
    query=0;
    destino=0
}

About ILDE

ILDE is a term to describe (intermediate level embedded Driver), which has CQL function blocks embedded.

We have open-source distributions for several languages:

  • ILDE-Lua.

  • ILDE-Pascal (Delphi, Lazarus).

  • ILDE-Python.

  • ILDE-Java.

  • ILDE-Dart (new Google language).

Example ILDE-Lua:

function Transaction()
  local DB = "c:\\MyDB\\"
  local Table = "Products"
  local code = "00001"
  local Product = "Rice"
  local Price ="3,45" -- comma is decimal delimiter
  local fields = code..Products..Price  --concat all to fields
  CSV_COMP_DB_execute(  add_data(DB ,Table ,fields)  )--ILDE Lua is similar to java
end
--call
Transaction()

Where to Apply the CSV Comp DB

For the uses that I have already developed, it was possible to create several applications:

  • Sites.

  • E-commerce.

  • Databases for web or desktop.

  • Supervisors for sensors and microcontrollers.

It’s easier than you think to extend DevOps practices to SQL Server with Redgate tools. Discover how to introduce true Database DevOps, brought to you in partnership with Redgate

Topics:
database ,csv ,database management systems ,rdbms ,database management

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}