Data Engineering Resources

Serverless Extraction and Processing of CSV Content From a Zip File With Zero Coding

This article demonstrates the implementation of a serverless function in AWS using Kumologica for unzipping and extracting CSV file content.

April 26, 2023

by Satheesh Valluru

· 5,376 Views · 2 Likes

The True Cost of Building Your Own Incident Management System (IMS)

This article outlines some of the key factors to consider while choosing whether to build or buy incident management software.

April 26, 2023

by Biju Chacko

· 3,921 Views · 1 Like

Benefits of Manual Penetration Testing

While automated pen testing has perks, manual pen testing is still beneficial. Manual pen testers can rely on hard-earned experience, prevent false positives, and more.

April 26, 2023

by Zac Amos

· 3,255 Views · 1 Like

A Complete Guide To Selenium Page Load Strategy

This blog will take a deeper look at Selenium Page Load Strategy, its different types, and how they can be implemented in automation code.

April 26, 2023

by Vipul Gupta

· 2,633 Views · 1 Like

Ethical Prompt Engineering: A Pathway to Responsible AI Usage

Ethical prompt engineering is the process of crafting input queries or prompts for AI models in a way that minimizes biases and promotes fairness.

April 26, 2023

by Navveen Balani

· 3,358 Views · 3 Likes

Go and Redis: Better Together

Learn valuable information presented at a talk from Conf42 Golang 2023, which was geared toward folks who are looking to get started with Redis and Go.

April 26, 2023

by Abhishek Gupta

CORE

· 14,498 Views · 2 Likes

Delta, Hudi, and Iceberg: The Data Lakehouse Trifecta

Get a detailed overview of Delta Lake, Apache Hudi, and Apache Iceberg as we discuss their data storage, processing capabilities, and deployment options.

April 26, 2023

by Andrey Gusarov

· 11,712 Views · 4 Likes

Processing Large Files With Repeating Elements and Header/Trailer Fields

How to use IBM App Connect Enterprise File Nodes and DFDL Parser to process very large files with repeating elements and header/trailer fields.

April 26, 2023

by Dave Crighton

· 3,058 Views · 1 Like

The Equivalence Rationale of Neural Networks and Decision Trees: Towards Improving the Explainability and Transparency of Neural Networks

Let's unravel a new approach to improving the explainability and transparency of neural networks using an equivalent decision tree to represent them.

April 25, 2023

by Priya Kumari

· 2,942 Views · 2 Likes

Streamlining Master Data Management With Snowflake and SnapLogic

Building a Master Data Management (MDM) solution simplifies data integration, improves data quality and governance, and enhances master data management and analytics.

April 25, 2023

by Rama Krishna Panguluri

· 4,153 Views · 3 Likes

Azure Data Box

Azure Data Box is a product offered by Microsoft Azure that helps organizations transfer large amounts of data securely and efficiently to and from Azure.

April 25, 2023

by Shripad Barve

· 4,082 Views · 2 Likes

Connection Pool High Availability With CockroachDB and PgCat

This article covers pooler high availability working with PgCat to provide a highly available configuration where clients can see multiple connections.

April 25, 2023

by Artem Ervits

CORE

· 3,795 Views · 1 Like

Frequently Faced Challenges in Implementing Spark Code in Data Engineering Pipelines

This article discusses some of the common challenges faced by data engineers in Pyspark applications and the possible solutions to overcome these challenges.

April 25, 2023

by Amlan Patnaik

· 7,125 Views · 2 Likes

Building a RESTful API With Java Spring Boot

Build powerful RESTful APIs using Java Spring Boot. Use our quick and easy tutorial for retrieving credit scores and setting up a basic REST API using Postman.

April 25, 2023

by Dylan Frankcom

· 5,384 Views · 1 Like

A Critical Detail About Kafka Partitioners

Understanding the role of a partitioner in the Kafka producer is important to understand to build effective Kafka applications.

April 25, 2023

by Bill Bejeck

· 4,688 Views · 2 Likes

Potential Benefits of Using Generative AI in the Development and Operations of IT Systems

Many organizations are experimenting with how generative AI can be used to develop and operate IT systems.

April 25, 2023

by Divakar R

· 5,749 Views · 1 Like

How To Use Cypress Intercept for Handling Network Requests

Deep diving into using Cypress intercept for handling network requests, let’s first understand the nuances of network testing while performing Cypress testing.

April 25, 2023

by Kailash Pathak

CORE

· 3,623 Views · 2 Likes