View unanswered posts | View active topics It is currently Fri Oct 19, 2018 4:25 am



Post new topic Reply to topic  [ 9 posts ] 
 New Webmail Dissector 
Author Message

Joined: Tue Nov 02, 2010 2:59 pm
Posts: 5
Reply with quote
Post New Webmail Dissector
Firstly, Gianluca excellent work done. I have been using Xplico recently after a friend suggested and I am mighty impressed.

Now I am not a very technical person so everyone will hav eto excuse me if my question is too trivial. I was wondering what needs to be done to add a new webmail dissector. One way would be to follow Xplico's architecture, the otehr could be to write our own dissector and store the results in Xplico's format in the relevant Pol/Sol folders along with updating the database entries. Am I right in assuming so?

Secondly, now if I want to write a webmail dissector. The input would me a PCAP file with tons of data. Firsatly I rearrange the packets. So say out of 10000 packets in my PCAP, all arranged, packet 2500- 2550 belong to my webmail. I can identify the first relevant packet using the msg HOST string (for instance mail.yahoo....), but how do I identify the end of that HTTP page? How do I identify the last packet. Once I do that, I can pass that data through my dissector and capture the info.

experts, please suggest!!!

Abhi


Tue Nov 02, 2010 3:08 pm
Profile
Site Admin

Joined: Wed Sep 16, 2009 10:09 pm
Posts: 394
Reply with quote
Post Re: New Webmail Dissector
Quote:
Firstly, Gianluca excellent work done.

Thanks. If you want to support us you can also make a donation.
Quote:
I was wondering what needs to be done to add a new webmail dissector.

Which webmail would like to add?
Quote:
One way would be to follow Xplico's architecture, the otehr could be to write our own dissector and store the results in Xplico's format in the relevant Pol/Sol folders along with updating the database entries.

For webmail you do not need to add a new dissector. There is already a webmail dissector and should be fine for most cases (making small changes), unless your webmail is very special.
Quote:
Am I right in assuming so?

As I said, depends on the webmail. Which webamil you want to implement?

Quote:
Secondly, now if I want to write a webmail dissector. The input would me a PCAP file with tons of data. Firsatly I rearrange the packets. So say out of 10000 packets in my PCAP, all arranged, packet 2500- 2550 belong to my webmail. I can identify the first relevant packet using the msg HOST string (for instance mail.yahoo....), but how do I identify the end of that HTTP page? How do I identify the last packet.

The approach is broadcaster. Continuing your reasoning then you should also consider all these issues:
  • HTTP pipeline
  • HTTP "chunked" transfer-coding
  • and more
In Xplico there is also a HTTP dissector which handles all these issues. In fact, webmail dissector depends by http dissector (how you can check with the -g command from the CLI).
Quote:
Once I do that, I can pass that data through my dissector and capture the info.

You will not have this requirement.

What you have to make is:
  • Locate the 'URL (in its all possible formats) that contains the email sent.
  • Locate the 'URL (in its all possible formats) that contains the email received / read
  • Write a python script that uses the page file as input (in the general body of the URL/POST) of the mail sent and extract the sender, recipients, and the body of the email. .
  • Write a python script that uses the page file as input (in the general the body of the URL request) of the email read/received and extract: the sender; all recipients; the body of the email.
When you do this, send this information by email or in this post and we'll walk you to integration.

Ciao.
Gianluca


Thu Nov 04, 2010 6:50 am
Profile WWW

Joined: Tue Nov 02, 2010 2:59 pm
Posts: 5
Reply with quote
Post Re: New Webmail Dissector
Hi Costa,

I am back after a long gap!!! Started looking into this again. Alright, I believe the best thing would be to add a new dissector following XPLICO's architecture. Lets say, I want to try out GMAIL (the old HTTP version and not the HTTPS version). The format is very very similar to Hotmail/Live.

Now, I think the following needs to be done -

1). In webmail.h adda hostname and service for gmail
2). In webmail.C make changes to checek for Hostname "Mail.google.com" and if found call the manipoulator "WebmailPei(WMAIL_SERVICE_GMAIL, pkt, FALSE)"
3). Change the analise.c to write a new manipulator for gmail "static pei *WMGMAIL(const pei *ppei)"
4). Change Analise.h and add the service type for gmail
5). write a Py script for gmail on the lines of wbm_live.py

Now, i did all that. I was able to successfully comiple it also. But when I load a gmail.pcap it doesnt decode anything.... I even wrote a small printf for debugging in the loop for webmail.c -

else if (strstr(msg->host, "mail.google.com") != NULL) {
/* send to manipulator */
WebmailPei(WMAIL_SERVICE_GMAIL, pkt, FALSE);
ins = TRUE;
printf("Checking host for gmail successful------!!!!");
}

but I dont get this test string printed in the CLI.Suppose I just make changes to the webmail.c (and webmail.h for the defines) and do nothing else, at least the test printf should work? Why isnt the code checking the host as mail.google.com and satisfying the IF condition?

I am attaching the pcap, webmail.c and webmail.h for your analysis. any help is muchhhhhhhhhhhhhh appreciated!!!

Really looking forward to some expedited help...

Cheers!
Abhi


You do not have the required permissions to view the files attached to this post.


Tue Jan 25, 2011 3:42 pm
Profile

Joined: Tue Nov 02, 2010 2:59 pm
Posts: 5
Reply with quote
Post Re: New Webmail Dissector
attaching the pcap again....


You do not have the required permissions to view the files attached to this post.


Tue Jan 25, 2011 3:48 pm
Profile
Site Admin

Joined: Wed Sep 16, 2009 10:09 pm
Posts: 394
Reply with quote
Post Re: New Webmail Dissector
Hi,
your pcap is not a real pcap file.


Wed Jan 26, 2011 6:40 pm
Profile WWW

Joined: Tue Nov 02, 2010 2:59 pm
Posts: 5
Reply with quote
Post Re: New Webmail Dissector
Hi,

I captured a new PCAP file via wireshark. I am unable to attach it here so sent an email. Please check the same.

Many Thanks.

Abhi


Fri Jan 28, 2011 8:09 am
Profile
Site Admin

Joined: Wed Sep 16, 2009 10:09 pm
Posts: 394
Reply with quote
Post Re: New Webmail Dissector
Hi,
I saw your code (.c and .h), but there are many errors and your code can not be compiled!
Please test (and compile) your code before request us help ;) .
Ciao.
Gianluca


Tue Feb 01, 2011 8:53 am
Profile WWW

Joined: Tue Nov 02, 2010 2:59 pm
Posts: 5
Reply with quote
Post Re: New Webmail Dissector
Hi,

Apologies for the bugs!!

My intention was to understand the integration process. I understand that the webmail dissector depends on the HTTP dissector, and the webmail 'fields' are extracted by the Python scripts. What I would really appreciate is to understand that if I have made a Python script, what are the steps for integration? Apologies for the incessant questions but if nothing else can you just let me know the files I would need to concentrate on? I will try to work my way up from there, as right now its an ocean of code out there so a little guidance would be greaaaaatttttlyyy appreciated!

Cheers!
Abhi


Tue Feb 01, 2011 5:35 pm
Profile
Site Admin

Joined: Wed Sep 16, 2009 10:09 pm
Posts: 394
Reply with quote
Post Re: New Webmail Dissector
Hi,
Quote:
1). In webmail.h adda hostname and service for gmail
2). In webmail.C make changes to checek for Hostname "Mail.google.com" and if found call the manipoulator "WebmailPei(WMAIL_SERVICE_GMAIL, pkt, FALSE)"
3). Change the analise.c to write a new manipulator for gmail "static pei *WMGMAIL(const pei *ppei)"
4). Change Analise.h and add the service type for gmail
5). write a Py script for gmail on the lines of wbm_live.py

yes, these are the integration steps.
In Xplico everything is a chain (in some cases it may also branching).
The Webmail chain is:
http(d)->webmail(d)->dispatcher(c)->manipulator(mwmail)->python->dispatcher(sqlite/cli/ximysql).
where:
    (d): dissectort module
    (c): embedded core functionality
    (mwmail): webmail manipulator
    (sqlite/cli/ximysql): dispatchers's type
The main activity is the python script. If you post your code (all: webmail.c/h, analise.c/h, wbm_gamil.py) we can help you.

Ciao.
Gianluca


Wed Feb 02, 2011 7:23 am
Profile WWW
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 9 posts ] 


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by Vjacheslav Trushkin for Free Forums/DivisionCore.