receiving sent from email address when opening email file attachment in android

I am developing an application that requires a custom file type to be attached to email messages. I have had no problem being able to open these custom file attachments in my application, or initiating a new message from my app and attaching my custom file to the ACTION_SEND Intent. But I am a bit stumped when I want to implement a "reply" function to files that have been opened by my app. What I want is the "Sent From" address from the email client so I can reply back using this Sent From address, as opposed to having to query the user for the address they user wants to reply to.

Any ideas would be greatly appreciated. I am thinking that this may not be possible.