DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
View Events Video Library
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Integrating PostgreSQL Databases with ANF: Join this workshop to learn how to create a PostgreSQL server using Instaclustr’s managed service

Mobile Database Essentials: Assess data needs, storage requirements, and more when leveraging databases for cloud and edge applications.

Monitoring and Observability for LLMs: Datadog and Google Cloud discuss how to achieve optimal AI model performance.

Automated Testing: The latest on architecture, TDD, and the benefits of AI and low-code tools.

Related

  • Mastering Persistence: Why the Persistence Layer Is Crucial for Modern Java Applications
  • Beyond Code: The Crucial Role of Databases in Software
  • How VAST Data’s Platform Is Removing Barriers To AI Innovation
  • Leveraging Query Parameters for Efficient Data Filtering in REST APIs

Trending

  • Information Security: AI Security Within the IoT Industry
  • Choosing the Appropriate AWS Load Balancer: ALB vs. NLB
  • Supercharging Productivity in Microservice Development With AI Tools
  • Embracing Reactive Programming With Spring WebFlux
  1. DZone
  2. Data Engineering
  3. Data
  4. Learning R: Hard Lessons

Learning R: Hard Lessons

Fair warning, there is no happy conclusion to this story. It’s a lesson about learning, not about solving a problem.

Grant Fritchey user avatar by
Grant Fritchey
·
Dec. 24, 15 · Opinion
Like (6)
Save
Tweet
Share
7.18K Views

Join the DZone community and get the full member experience.

Join For Free

I’ve always found the best way to learn a new programming language is to start building stuff, solving problems, using the language. Even if you do things badly or inefficiently, you’re figuring out how to put the language to use. I tried the same thing with R.

Fair warning, there is no happy conclusion to this story. It’s a lesson about learning, not about solving a problem.

After poking at the R language for a little while, I decided I was ready to solve a problem. I have a fantastic idea for demonstrating the usefulness of the language specifically for DBAs. I won’t go into what it is here because I’m still hoping to solve this problem and it will provide a fantastic blog post. Anyway, I have a very good understanding of the problem (or so I thought) I’m looking to solve, so, what the heck, let’s throw R at it.

First thing I learned was how to connect R up to a SQL Server database. Easy. Then it’s query the database to create a data frame. Easy. Referencing columns and rows from the data frame, again, really simple. Spit out some interesting information about the data such as density or a histogram? A couple of Bingle searches later, easy. Here’s a starter (formatted as text, my code formatter in the blog doesn’t have an R language choice):

library(RODBC)

#set up queries
sqloutput <-"SELECT  rd.FirstVal,
        rd.SecondVal
FROM    dbo.Mytable AS rd;"

#connect to server
dbhandle <-
  odbcDriverConnect(
    'driver={SQL Server};server=WIN-3SRG45GBF97\\dojo;database=Testing;trusted_connection=true'
  )

#load tables
mydata <- sqlQuery(dbhandle,sqloutput)

#do something
density(sqloutput[,1])
density(sqloutput[,2])

Seriously, this really is an easy enough language to learn. It’s kind of quirky, but it’s easy to understand as long as you are quick to look up syntax(it really is quirky). So what’s the problem?

Algorithms.

But that’s not even true. For example, I can pretty quickly run a Chi Square Independence test against my data like this:

chisq.test(relateddata)

Then you get the output and all hell breaks loose…. or… more accurately, no hell breaks loose. You’re looking at some data and trying to figure out what it means. Suffice to say, I finally realized that I had the wrong algorithm. More web searches. Questions posted to various forums, defining my problem (as I saw the definition), which lead me to the fact that what I really needed to solve my problem was not to run standard comparison tests, but to compare continuous distributions. Awesome. Fixed that algorithm problem, right?

Wrong.

That’s because algorithms are not the problem… the only problem. The real problem is data preparation. A lot of the examples you’ll read online are very straight forward with nice neat data sets. That’s because they were carefully groomed and prepared. Here I am looking at the wooly wild real data and I’m utterly lost in how to properly prepare this so that it’s appropriately set up as a continuous distribution(or a distribution at all). WOOF! The reason this is so hard is because I actually don’t understand the data fundamentals of the problem I’m trying to solve in exactly the way needed to solve the problem. More cogitation is necessary.

My lesson. I’m going to continue learning R syntax. I’m absolutely going to keep studying the mathematics in order to better understand the algorithms. But I’ve got to also spend time figuring out how to groom the data. It’s pretty clear that is one of the biggest challenges to making this stuff work.

Oh, and make sure you close your connection to the database:

#cleanup
odbcClose(dbhandle)


R (programming language) Database Data (computing)

Published at DZone with permission of Grant Fritchey, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Mastering Persistence: Why the Persistence Layer Is Crucial for Modern Java Applications
  • Beyond Code: The Crucial Role of Databases in Software
  • How VAST Data’s Platform Is Removing Barriers To AI Innovation
  • Leveraging Query Parameters for Efficient Data Filtering in REST APIs

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: