Rathnadevi Manivannan

[Deactivated] [Suspended]

Senior Technical Writer at Treselle Systems

Joined May 2017

http://treselle.com/blog

Stats

Reputation:	1007
Pageviews:	843.4K
Articles:	16
Comments:	5

Articles
Comments

Articles

Ingesting IoT Sensor Data Into S3 With an RPI3

StreamSets Data Collector Edge is a lightweight agent used to create end-to-end data flow pipelines. We'll use it help stream data collected from a sensor.

December 30, 2017

· 10,165 Views · 4 Likes

Sensor Data Quality Management Using PySpark and Seaborn

Learn how to check data for required values, validate data types, and detect integrity violation using data quality management (DQM).

December 2, 2017

· 10,914 Views · 5 Likes

Import and Ingest Data Into HDFS Using Kafka in StreamSets

Learn about reading data from different data sources such as Amazon Simple Storage Service (S3) and flat files, and writing the data into HDFS using Kafka in StreamSets.

October 26, 2017

· 23,119 Views · 5 Likes

API Response Tracking With StreamSets, Elasticsearch, and Kibana

Learn how to track JSON response data from a RESTful API using Elasticsearch and Kibana to capture and visualize the alerts.

October 15, 2017

· 11,956 Views · 3 Likes

Handling Imbalanced Data With R

Imbalanced data is a huge issue. With imbalanced data, accurate predictions cannot be made. Learn how to tackle imbalanced classification problems using R.

October 9, 2017

· 40,223 Views · 3 Likes

Data Analysis Using Apache Hive and Apache Pig

Learn about loading and storing data using Hive, an open-source data warehouse system, and Pig, which can be used for the ETL data pipeline and iterative processing.

August 18, 2017

· 25,751 Views · 5 Likes

Using Airflow to Manage Talend ETL Jobs

Learn how to schedule and execute Talend jobs with Airflow, an open-source platform that programmatically orchestrates workflows as directed acyclic graphs of tasks.

August 3, 2017

· 21,204 Views · 4 Likes

Using NGINX With GeoIP MaxMind Database to Fetch Geolocation Data

Learn how to find the geographical location of a user using their IP address by just configuring NGINX with GeoIP MaxMind databases — without doing ANY coding!

July 29, 2017

· 15,727 Views · 4 Likes

Database Performance Testing With Apache JMeter

Learn how to construct your database performance testing plan with all the most important elements, including a thread group, JDBC request, and summary report.

July 27, 2017

· 36,168 Views · 3 Likes

Data Normalization and Filtration Using Drools

Learn about normalizing and filtering data using Drools by looking at an example of oil well drilling datasets from Arkansas and Oklahoma.

July 20, 2017

· 14,079 Views · 4 Likes

Data Flow Pipeline Using StreamSets

Learn about configuring JDBC Query Consumer, performing JDBC lookup with multiple tables, creating a data flow pipeline, and monitoring the stage and pipeline stats.

July 18, 2017

· 15,962 Views · 7 Likes

Protractor With Cucumber

Learn how to configure the Protractor testing framework to use with the Cucumber Behavior-Driven Development framework for testing AngularJS applications.

July 11, 2017

· 27,234 Views · 4 Likes

Pivoting and Unpivoting Multiple Columns in MS SQL Server

In this article, we'll discuss converting values of rows into columns (PIVOT) and values of columns into rows (UNPIVOT) in MS SQL Server.

Updated July 9, 2017

· 251,455 Views · 5 Likes

Apache Spark Performance Tuning – Degree of Parallelism

Today we learn about improving performance and increasing speed through partition tuning in a Spark application running on YARN.

June 30, 2017

· 100,275 Views · 8 Likes

Apache Spark on YARN: Resource Planning

Apache Spark is an in-memory distributed data processing engine and YARN is a cluster management technology. Learn how to use them effectively to manage your big data.

June 28, 2017

· 36,744 Views · 8 Likes

Apache Spark on YARN – Performance and Bottlenecks

In this series, we learn about performance tuning and fixing bottlenecks in high-level APIs with an Apache Spark application on YARN.

June 27, 2017

· 30,353 Views · 12 Likes

Comments

Pivoting and Unpivoting Multiple Columns in MS SQL Server

Apr 30, 2018 · Rathnadevi Manivannan

Thanks Sandeep

Data Normalization and Filtration Using Drools

Apr 10, 2018 · Rathnadevi Manivannan

Please follow the below steps to test it locally:

Download raw data of Arkansas and Oklahoma.
Move the raw data into MS SQL. The source code is available in - https://github.com/treselle-systems/data_normalization_and_filtration_using_drools.
Change sql server connection url, username, and password in src/common.properties.
Change the table names in input & output queries in the property file.

Using Airflow to Manage Talend ETL Jobs

Jan 08, 2018 · Rathnadevi Manivannan

Hi Ranga Nathan,

Thanks for your review comments.

The scripts required to run the Talend jobs are mentioned in the DAG files. SQL has nothing to do with DAG and scheduling.

Crime Analysis Using H2O Autoencoders (Part 1)

Jan 03, 2018 · Rathnadevi Manivannan

Thanks for your comment

Data Analysis Using Apache Hive and Apache Pig

Sep 04, 2017 · Rathnadevi Manivannan

You are welcome