Dear Xplico Team:

Configuration:
1. One Case with One or Multiple Sessions
2. PCAP is approximately 4 days of capture with multiple protocols
3. PCAP has been sliced and diced multiple ways and either copied directly to the sol_#/new/ folder or streamed through PCAP to IP):
3.a. One large 60GB PCAP
3.b. Split pcap by 6 hour blocks of ranging size from 100MB to 22GB
3.c. Split pcap of 1000000 frames ranging in size from 400 to 750MB
4. Checksum on and off
5. The following dissectors off: VLAN, NNTP, ARP, PPPoE, IPv6, L2TP, IEEE80211
6. /opt/xplico/ is mounted to a dedicated RAID0 array of 4x 2TB disks.

Problem:
Xplico moves through the phases of EMPTY, BEGIN DECODING, and DECODING, but on occasion does not reach DECODING COMPLETE. I am unable to consistently reproduce the problem. I am able to pass the pause/hang using two methods:

A. Using the command line "service xplico restart" (not ideal, because it re-processes and creates duplicate entries for information already processed in that session).
B. Creating a new session, moving the pcap from /sol_#/raw/ to /sol_#+1/new/ and deleting the old session.

This problem occurred in the previous version, and was sometimes (but not always) accompanied by mime_dump.pyc crashes. In this version I am not getting any alerts that mime_dump.pyc has crashed.

I am unable to send you any segment of the PCAPs because they contain sensitive information.

Sincerely,

Eric

Follow Up. The log file is filled with these messages when it hangs.

.
.
.
15:04:24 ERROR: Xplico or a Manipulator is dead!

15:04:24 ERROR: Xplico or a Manipulator is dead!

15:04:24 ERROR: Xplico or a Manipulator is dead!

15:04:24 ERROR: Xplico or a Manipulator is dead!

15:04:25 ERROR: Xplico or a Manipulator is dead!

Hi Eric,

• the version of Linux distribution that you use?
  
• the other log (any manipulator generates a log file, in the same dir of the xplico log dir)
  
• can you send us (privately) the pcap set? we will use it only to debug and fix the issue.
  
• otherwise can you give us the access to the data (pcap) in a your machine, with a ssh connection?

Ciao.
Gianluca

Hi!

I have exactly the same symptoms:

My status with Xplico: Noob
Xplico version 1.1.2
Linux version: Kali Linux 2016.01 - Live USB Persistence mode
Installed Xplico using apt-get install Xplico
Everything is default default, no changes to configs or whatsoever.

Pcap is a 9 MB Pcap made with Wireshark (saved as tcp dump pcap).
After uploading pcap via web gui the status stays on "decoding" forever.
Dema log gets filled up with "Xplico or a Manipulator is dead!" message.
htop shows 6 /opt/xplico/ processes re-spawning every 5 or 6 seconds (see image).


Running Xplico from cli also gives errors.
bin/xplico -c cfg/xplico_cli.cfg -m pcap -f Downloads/test.pcap
It shows only it has found the config file but nothing else. Looking at the Xplico log in /opt/xplico/tmp shows the following errors:
19:44:35 [CORE]{c}-ERROR: Can't load module /opt/xplico/bin/modules/dis_tcp_grb.so: undefined symbol: ndpi_protocol2name
19:44:35 [CORE]{c}-FATAL: Load modules failed

Any suggestions on how to fix?

Tnx!

Hi "Rodger Moore",
the issue related to the CLI execution is the nDPI library (linking).

Which version of nDPI are you using?

Ciao.
Gianluca

Ok good question! Seems on the new Kali Rolling Release nDPI is not installed at all. So what I tried is:

apt-get install dh-autoreconf tcpdump tshark apache2 php5 php5-sqlite build-essential perl libzip-dev libpcap-dev libsqlite3-dev php5-cli libapache2-mod-php5 libx11-dev libxt-dev libxaw7-dev python3 python3-httplib2 python3-psycopg2 sqlite3 recode sox lame libnet1 libnet1-dev binfmt-support libssl-dev
mkdir xbuild
cd xbuild/
svn co https://svn.ntop.org/svn/ntop/trunk/nDPI
cd nDPI/
./configure
NOT WORKING BECAUSE IT WAS AN EMPTY DIR -> nDPI moved to Git

git clone https://github.com/ntop/nDPI.git
cd nDPI/
./autogen.sh
./configure --prefix=/usr
make
make install
cd /opt/xplico/bin
./xplico -m pcap ~/Documents/test.pcap

Still the same error...

Any suggestions?

When I ldd bin/xplico I don't see any link to libndpi... Is this correct?


I installed the nDPI 1.7 branche from Github. Just double checking if ndpi_protocol2name is present in /usr/lib/libndpi.so.1.0.0 and this seems the case:



Suggestions?

Hi Rodger,
the original source code of Xplico uses nDPi statically. This is why nDPi must be compiled before Xplico and its source code must be in the same level (parent dir) of Xplico:

the modules (dissectors) which use nDPI are:

• tcp_grb
• udp_grb
• tcp_ca (added on xplico 1.1.2)
• udp_ca (added on xplico 1.1.2)

If you are using a modified version of Xplico where the nDPI is linked dynamically to Xplico try to see where the nDPi has been installed in your distro, for example seeing where xplico links the nDPi library.


I hope it can help you.

Ciao.
Gianluca

Hi Gianluca,

Thanks for your answer. So it seems to be an error in the Kali repository, Ubuntu installation using your repo works fine. I posted a bugtracker here : https://bugs.kali.org/view.php?id=3065. Hope it gets solved soon.

Cheers.