Android Email Extraction to .eml
Join the DZone community and get the full member experience.
Join For Freesometimes the android ecosystem is a little lacking with tool support; for instance i needed to extract a set of sent items from a pop3 mailbox – the stock mail client only allows you to perform 3 actions: delete, mark as unread or favourite.
armed with the android sdk, some sql queries and a groovy script we’ll see how it’s possible to recover email to rfc822 .eml files.
email storage
the stock com.android.email client stores message headers and their bodies in two sqlite databases and the attachmentprovider stores attachments on disk.
the source code is available from https://android.googlesource.com/platform/packages/apps/email – in this case it was for a samsung phone running 4.1.2 so i checked out the jb-mr0-release branch to inspect the code when required.
backup with adb
adb is the android debug bridge , a debugging tool that is part of the android sdk platform tools . executing ‘ adb usb ‘ (re)starts the adbd daemon listening to usb connections.
the next step is to select developer options from the system section of the settings list (figure 1) and then enable usb debugging on the device (figure 2).
then connect to the device using a usb cable and execute ‘ adb backup -f mybackup.ab -all ‘ (you can also be more selective with the package you want, e.g. ‘ adb backup -f mybackup.ab com.android.email ‘). adb will prompt you to unlock the phone and permit the backup to proceed.
inflating the backup
an android backup is a zlib deflated tar file – thanks to
http://nelenkov.blogspot.jp/2012/06/unpacking-android-backups.html
for the hint that it can be reinflated with the following command:
dd if=mybackup.ab bs=24 skip=1|openssl zlib -d > mybackup.tar
note that the install of openssl i had on osx wasn’t compiled with zlib support, so i ran this through an ubuntu vagrant vm to inflate the tar file.
examining the backup
unpacking the tar file will give you a folder structure as shown in figure 3; in this case the sqlite databases are in db and photo attachments taken with the phone are in f .
exploring the databases
sqlitebrowser is a good tool for browsing sqlite database and features in the following screenshots.
the current script restricts the messages processed to a single mailbox for one account. to determine the mailboxkey requires locating the target account from the account table (figure 4) and then isolating the desired mailbox for that account within the mailbox table (figure 5).
you can browse the data or execute the following query to get a list of mailboxes and their corresponding accounts:
select a.displayname as accountname, m._id as mailboxkey, m.displayname as mailboxname
from account a, mailbox m
where m.accountkey = a._id
if you don’t want to limit the extraction to a single mailbox or account then you can remove the where clause from the first select query.
email extraction
being comfortable with the rfc822 specs & javamail (i built an imap extension for alfresco in 2006), i decided it would be easier to reconstruct a mimemessage using sql & groovy than attempt to adapt the android source code to run off the device or build a custom app to run in the emulator.
shortcomings
as a first cut that was good enough for my purposes, there are a few deficiencies:
1. it is set to default the sender field rather than converting the value of
message.fromlist
and using the
addfrom
method
2. addresses only use the address rather than the label
3. the body handling only uses plain text rather than alternative multiparts
4.
body.textreply
is separated from the
body.textcontent
with a separator line of 25 dashes; it does not attempt to reconstruct the header information of the preceding message in the thread
5. the script does not handle attachments – this was a conscious decision as whilst the camera photos were in
com.android.email/f
, the other ‘raw’ attachments were not in
com.android.email/1.db_att/
as per the javadoc of the
attachmentprovider
class
as an initial hint, to reconstruct the attachments you would need to use mimemultipart , create the mimebodypart objects starting with a query like:
"""select filename, mimetype, size, contentid, contenturi, encoding from attachment where messagekey = ${msgkey}"""
addresses
the script provides one utility method to convert a string-ified list where records are separated by the
soh
character (ascii 1) and email addresses and their corresponding display name are separated by the
stx
(ascii 2) character.
the processing consists of iterating through the rows of the message table result set, for each row the message body is obtained from a separate sqlite database and then a javamail mimemessage is constructed and output to a file.
troubleshooting
the script cannot cause data loss on your device as it is operating on a backup. if you want to experiment first you may like to set a limit (e.g.
limit 10
) on the first
select
query to reduce the number of messages that it retrieves.
also, a simple way of viewing the output within e.g. the groovy console is to use
msg.writeto(system.out)
instead of creating the .eml file.
get the script
the usual disclaimers apply that the script is without warranty, support etc. – get it from github: https://gist.github.com/rbramley/65261127dfb857b03bb6
Published at DZone with permission of Robin Bramley, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Trending
-
Application Architecture Design Principles
-
What Is JHipster?
-
Incident Response Guide
-
Best Practices for Securing Infrastructure as Code (Iac) In the DevOps SDLC
Comments