View unanswered posts | View active topics It is currently Fri Nov 16, 2018 1:49 am



Post new topic Reply to topic  [ 7 posts ] 
 How xplico act against iteration traffic? 
Author Message

Joined: Sat Oct 10, 2009 10:04 am
Posts: 38
Reply with quote
Post How xplico act against iteration traffic?
I have 2M traffic in pcap file format. I iterated it 5000 with a traffic generator software for testing xplico under high rate traffic (because I do not access to high rate traffic for example 80M).

But when I check volume of captured and decoded traffic by xplico CLI, it was about 20M not 2M*5000~10G

I check decoded files. I saw, for example google logo (picture in google.com site) exists only one sample, but pictures in yahoo index page(yahoo.com) exists many(duplicated).

My question is:

xplico categorize decoded files based on source IP & protocol when is ran CLI.
Can xplico understand duplicated files in traffic? How?
For example if I see google index page twice, can Xplico understand duplicate traffic & generate only one logo? if yes, why for some sites(for example yahoo.com) can not understand duplicated traffic and generate duplicated pictures?


Fri Dec 04, 2009 7:13 pm
Profile
Site Admin

Joined: Wed Sep 16, 2009 10:09 pm
Posts: 394
Reply with quote
Post Re: How xplico act against iteration traffic?
Quote:
xplico categorize decoded files based on source IP & protocol when is ran CLI.

No.
Quote:
Can xplico understand duplicated files in traffic? How?

No.

I have two hypotheses for the problem you reported.
  • a) Http dissectort bug
  • b) traffic generator software + basic pcap: mistake
a) Http dissectort bug
Possible bug in the generation of temporary file name (I will check).

b) traffic generator software + basic pcap: mistake
HTTP is based on TCP. A TCP flow is defined uniquely by (ip sorce, tcp source port, ip destination, tcp destination port) and (more important) time.
Moreover, each TCP flow (description trivial) begins with SYN packet and ends with a FIN packet or reset, and ach stream has a precise sequence of "sequence numbers".
If a tcp flow (with syn and fin) is replicated twice in the same time interval (the same timestamps in pcap) then the flows are not two but one.
If a tcp flow (without syn and fin) is replicated twice in the different time interval then the flow is only one, because the sequence numbers are replicated and then as if there were packets retransmitted.
These are just some examples.
It would be useful to have the original pcap and also the pcap product of your tool to understand the situation.

Ciao.
Gianluca


Sun Dec 06, 2009 11:25 am
Profile WWW

Joined: Sat Oct 10, 2009 10:04 am
Posts: 38
Reply with quote
Post Re: How xplico act against iteration traffic?
I attached cap file.

For testing, I connect two computers directly by one Ethernet cable so xplico only gets iterated traffic.

There is a wonderful thing:
I iterated cap file 4365 iterations. I expected Xplico generates about 5G traffic but:
  • When I use CLI, it generates about 20M
  • When I use web interface in live acquisition mode, it generate about 1G

Is there different between CLI and web interface in capturing & decoding traffic? Why volume of decoded files in CLI & web interface modes have different? In other words, when using web interface mode(live acquisition), volume of decoded files is very larger than using CLI mode? Is this related to disp_cli.so?


You do not have the required permissions to view the files attached to this post.


Thu Dec 10, 2009 10:46 am
Profile

Joined: Wed Sep 16, 2009 10:45 pm
Posts: 128
Reply with quote
Post Re: How xplico act against iteration traffic?
Which xplico.cfg file are you using? Could you paste too its content?


Thu Dec 10, 2009 11:48 am
Profile

Joined: Sat Oct 10, 2009 10:04 am
Posts: 38
Reply with quote
Post Re: How xplico act against iteration traffic?
I use xplico_cli.cfg when using CLI.

When using web interface, do not need to get xplico.cfg, it can get it form default path.


You do not have the required permissions to view the files attached to this post.


Thu Dec 10, 2009 3:18 pm
Profile
Site Admin

Joined: Wed Sep 16, 2009 10:09 pm
Posts: 394
Reply with quote
Post Re: How xplico act against iteration traffic?
Hi,
the only differences between XI and CLI are the capture dissector and the dispatcher.
I suppose you used the CLI in live mode: ./xplico -m rltm -i eth0 .
The CLI (in live mode) use rtlm capture dissector (cap_rltm.so module) and as dispatcher: cli (disp_cli.so module).
The XI (in live mode) use rltm_pol capture dissector (cap_rltm_pol.so module) and as dispatcher: lite (disp_lite.so module).
The main difference (considering your problem) between rtlm capture dissector and rltm_pol capture dissector is that rltm_pol create a pcap file of all acquired data, this file is in /opt/xplico/pol_<>/sol_<>/raw .
The main differences between cli dispatcher and lite dispatcher are that cli removes all files that contain HTTP header request and HTTP header responce and lite dispatcher creates for each reconstructed content a xml file (info.xml) that describes the reconstructed flows (pcap file, ip, port, ...).
So it is normal that the data extracted from XI are greater than those of the CLI. But I think in your case there is something abnormal. Give me time to see your pcap.

NOTE: The rltm_pol module can not be used in CLI.


Thu Dec 10, 2009 8:28 pm
Profile WWW
Site Admin

Joined: Wed Sep 16, 2009 10:09 pm
Posts: 394
Reply with quote
Post Re: How xplico act against iteration traffic?
Hi rp_exploit,
I had time to review your pcap.
In your pcap there are 38 TCP connections, only thirteen of these connections have the SYN packet and only thirteen of these connections have the FIN packet.
The connections that have at least one packet SIN or FYN are only 15.
With Wireshak try to merge (in append packet) two time yuor pcap and you will understand why iterate that pcap is not the right thing to do.
Attachment:
merge_tcp.pcap.gz

In this attach you can find a TCP stream (from your pcap) repeated twice. If you use Wireshark and make a "Flow TCP stream" you will see only the data from packet 1 to 101, as if the packages from 102 to 202 did not exist. For this, your test does not have the outcome you expect.
For yuor test, you must use a tool that changes the IP (at least one of the two) every iteration. I do not know very well Tcpreplay, but I think may be useful.

Ciao.
Gianluca


You do not have the required permissions to view the files attached to this post.


Mon Dec 14, 2009 7:25 pm
Profile WWW
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by Vjacheslav Trushkin for Free Forums/DivisionCore.