DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
Building Scalable Real-Time Apps with AstraDB and Vaadin
Register Now

Trending

  • Application Architecture Design Principles
  • What Is JHipster?
  • Incident Response Guide
  • Best Practices for Securing Infrastructure as Code (Iac) In the DevOps SDLC

Trending

  • Application Architecture Design Principles
  • What Is JHipster?
  • Incident Response Guide
  • Best Practices for Securing Infrastructure as Code (Iac) In the DevOps SDLC
  1. DZone
  2. Data Engineering
  3. Databases
  4. Android Email Extraction to .eml

Android Email Extraction to .eml

Robin Bramley user avatar by
Robin Bramley
·
Jun. 13, 14 · Interview
Like (0)
Save
Tweet
Share
3.29K Views

Join the DZone community and get the full member experience.

Join For Free

sometimes the android ecosystem is a little lacking with tool support; for instance i needed to extract a set of sent items from a pop3 mailbox – the stock mail client only allows you to perform 3 actions: delete, mark as unread or favourite.

armed with the android sdk, some sql queries and a groovy script we’ll see how it’s possible to recover email to rfc822 .eml files.

email storage

the stock com.android.email client stores message headers and their bodies in two sqlite databases and the attachmentprovider stores attachments on disk.

the source code is available from https://android.googlesource.com/platform/packages/apps/email – in this case it was for a samsung phone running 4.1.2 so i checked out the jb-mr0-release branch to inspect the code when required.

backup with adb

adb is the android debug bridge , a debugging tool that is part of the android sdk platform tools . executing ‘ adb usb ‘ (re)starts the adbd daemon listening to usb connections.

the next step is to select developer options from the system section of the settings list (figure 1) and then enable usb debugging on the device (figure 2).

figure 1 - system settings

figure 1 – system settings

figure 2 - developer options

figure 2 – developer options

then connect to the device using a usb cable and execute ‘ adb backup -f mybackup.ab -all ‘ (you can also be more selective with the package you want, e.g. ‘ adb backup -f mybackup.ab com.android.email ‘). adb will prompt you to unlock the phone and permit the backup to proceed.

inflating the backup

an android backup is a zlib deflated tar file – thanks to http://nelenkov.blogspot.jp/2012/06/unpacking-android-backups.html for the hint that it can be reinflated with the following command:
dd if=mybackup.ab bs=24 skip=1|openssl zlib -d > mybackup.tar

note that the install of openssl i had on osx wasn’t compiled with zlib support, so i ran this through an ubuntu vagrant vm to inflate the tar file.

examining the backup

unpacking the tar file will give you a folder structure as shown in figure 3; in this case the sqlite databases are in db and photo attachments taken with the phone are in f .

figure 3 - expanded tar

figure 3 – expanded tar

exploring the databases

sqlitebrowser is a good tool for browsing sqlite database and features in the following screenshots.

the current script restricts the messages processed to a single mailbox for one account. to determine the mailboxkey requires locating the target account from the account table (figure 4) and then isolating the desired mailbox for that account within the mailbox table (figure 5).

figure 4 - account table structure

figure 4 – account table structure

figure 5 - mailbox table structure

figure 5 – mailbox table structure

you can browse the data or execute the following query to get a list of mailboxes and their corresponding accounts:

select a.displayname as accountname, m._id as mailboxkey, m.displayname as mailboxname
from account a, mailbox m
where m.accountkey = a._id

if you don’t want to limit the extraction to a single mailbox or account then you can remove the where clause from the first select query.

email extraction

being comfortable with the rfc822 specs & javamail (i built an imap extension for alfresco in 2006), i decided it would be easier to reconstruct a mimemessage using sql & groovy than attempt to adapt the android source code to run off the device or build a custom app to run in the emulator.

shortcomings

as a first cut that was good enough for my purposes, there are a few deficiencies:
1. it is set to default the sender field rather than converting the value of message.fromlist and using the addfrom method
2. addresses only use the address rather than the label
3. the body handling only uses plain text rather than alternative multiparts
4. body.textreply is separated from the body.textcontent with a separator line of 25 dashes; it does not attempt to reconstruct the header information of the preceding message in the thread
5. the script does not handle attachments – this was a conscious decision as whilst the camera photos were in com.android.email/f , the other ‘raw’ attachments were not in com.android.email/1.db_att/ as per the javadoc of the attachmentprovider class

as an initial hint, to reconstruct the attachments you would need to use mimemultipart , create the mimebodypart objects starting with a query like:

"""select filename, mimetype, size, contentid, contenturi, encoding
from attachment
where messagekey = ${msgkey}"""

addresses

the script provides one utility method to convert a string-ified list where records are separated by the soh character (ascii 1) and email addresses and their corresponding display name are separated by the stx (ascii 2) character.

the processing consists of iterating through the rows of the message table result set, for each row the message body is obtained from a separate sqlite database and then a javamail mimemessage is constructed and output to a file.

troubleshooting

the script cannot cause data loss on your device as it is operating on a backup. if you want to experiment first you may like to set a limit (e.g. limit 10 ) on the first select query to reduce the number of messages that it retrieves.
also, a simple way of viewing the output within e.g. the groovy console is to use msg.writeto(system.out) instead of creating the .eml file.

get the script

the usual disclaimers apply that the script is without warranty, support etc. – get it from github: https://gist.github.com/rbramley/65261127dfb857b03bb6

Database Android (robot)

Published at DZone with permission of Robin Bramley, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Trending

  • Application Architecture Design Principles
  • What Is JHipster?
  • Incident Response Guide
  • Best Practices for Securing Infrastructure as Code (Iac) In the DevOps SDLC

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com

Let's be friends: