Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

IO Ninja — Programmable Terminal Emulator/Sniffer

DZone's Guide to

IO Ninja — Programmable Terminal Emulator/Sniffer

IO Ninja is a programmable terminal emulator and sniffer you can use for device interaction and other low-level IO tasks. Here's some of what IO Ninja can do, with a bit of history behind it.

Free Resource

Transform incident management with machine learning and analytics to help you maintain optimal performance and availability while keeping pace with the growing demands of digital business with this eBook, brought to you in partnership with BMC.

Today I would like to tell  about an interesting tool that can be used by IT-professionals, system administrators, network security experts, penetration testers, and software developers working on device interaction and other low-level IO tasks.

The tool I am referring to is a programmable terminal emulator and sniffer called I/O Ninja (from now on I will omit the word “emulator” and simply say “terminal”). I suspect that the definition itself — “terminal/sniffer” — may sound unusual, if not weird. So I’m going to start with the story of its creation.

Bits of History

The very first product made by our company, Tibbo Technology, was the EM100 serial-to-Ethernet module.

A large portion of the development process revolved around designing and implementing network communication protocols for interaction between the module and a virtual serial port of a PC.

This was arduous, hard work. In our cruel world nothing works as intended — well, at least not immediately. So we spent long months of designing and debugging our communications. Here is the list of software tools that we employed:

  • Serial terminal;
  • TCP terminal;
  • UDP terminal;
  • Serial monitor;
  • Ethernet sniffer.

When designing and debugging our product we needed to:

  • Listen and accept incoming TCP connection;
  • Send broadcast UDP packets;
  • Receive replies to UDP packets from multiple nodes;
  • Send binary data over TCP , UDP, and serial links;
  • Display the HEX representation of sent and received binary data.

We didn’t have any problems with Ethernet and serial data monitors — even free products available at that time fully satisfied our needs. Things weren’t as smooth on the terminal side of our work.

We found a plethora of free software terminal, but they all had their limitations. First, most of them were not that great at working with binary data. Some did not implement the binary mode at all, and those who did weren’t very convenient. Terminal software we tried also did not have a good HEX editor supporting select, copy, paste, undo, and redo operations. We also found that almost no terminal utilities allowed incoming TCP connections, and none offered support for broadcast UDP packets.

It was after my unsuccessful quest for the required toolset that I’ve got the idea to create a universal all-in-one software package that would incorporate all of the IO facilities we so desperately needed in our work. This is when I thought to myself: Wouldn’t it be awesome if this new software was scriptable and allowed adding custom protocol analyzers, syntax highlighters, and auto-response scripts (that wait for a packet, analyze it, generate and send a reply, wait again, and so on)?

Well, the scripting part of my dream remained a dream for a very long time. The first version of my software was “hardcoded”, and it still proved to be immensely useful. This first release was a small utility for sending and receiving binary data, with a decent HEX editor, HEX viewer, the ability to accept TCP connections, as well as work with broadcast UDP packets. This utility was uncreatively named “SockTerm”. Here is how it looked:

Besides supporting “missing” socket modes, this utility demonstrated the great convenience of its innovative logging approach (which I since patented). The log was presented as a seamless HEX sheet with packets of the same kind stitched together.

I kept working on this utility in my spare time for a year or so. I improved the logging module and implemented a fully functional logging engine. When the engine was ready I recognized that it was suitable for use not only in terminals, but in sniffers and monitors as well. Convinced that I was on the right track, I kept working on the project and, after several more months of development we released our first official version of the product, now called I/O Ninja:

At this stage, I/O Ninja already allowed us to work with all the transports we needed (serial, TCP, UDP), and it did so in two capacities: as a terminal, and as a non-intrusive sniffer/monitor. Different kinds of I/O session were implemented as plugins, which were written, like the software itself, in C++. In theory, this plugin architecture allowed us to extend I/O Ninja’s functionality as much as we needed. We even through about publishing a plugin SDK and thus allow “advanced” users to create their own plugins. In reality we never got around to doing this. Why? Because we wanted to aim higher and implement real programmability.

What we wanted to do was to make I/O Ninja scriptable. Question was, what scripting language should we choose? Should it be Python? Lua? JavaScript? Some of its twins?

Well, just like we had a dream of having a scriptable terminal, we had a vision of what a perfect language for this terminal would look like. And two of the biggest “must haves” on the wish list for this perfect scripting language were:

  1. High level of compatibility with C (both on the source and binary level), and…
  2. Safe pointer arithmetic.

It’s easy to see why these two features are so desirable. C is the standard choice for system, network, and other low-level programming. You can find the C implementation of any protocol, any checksum calculation algorithm, any cryptographic algorithm. On top of that, C-structs and pointer arithmetic offer the best way of dealing with binary buffers!

Now add to this the ability to run that code safely and not be bugged by pointer-induced problems — and you’ve got yourself a perfect language for writing I/O-related scripts.

I looked around in hopes of finding such a language. To my surprise, nothing was available. The scripting language I needed simply did not exist! I know, right? There are almost 700 languages on the Wikipedia page listing all notable programming languages, and none offered safe-scripting, pointer-supporting version of C!

To fix this ridiculous situation I have invented the Jancy scripting language. While featuring a high level of source and binary compatibility with C, Jancy provides safe pointers and safe pointer arithmetic. It also has a plethora of other delicious IO-related features such as built-in regular expression-based Lexer generator, scheduled function pointers, reactive programming, and much much more.

Once we’ve got the first prototype of Jancy up and running, we immediately started using it as a scripting engine for the new generation of our I/O Ninja software, which was released in late 2014.

Conclusion

The biggest advantage of IO Ninja is, by far, its programmability — all of its sessions are implemented as open-source scripts, giving you the opportunity to fine-tune them to your liking (or roll out your own sessions for your own transports and protocols). Even without writing a single line of code, IO Ninja is an extremely useful tool offering you:

  • All-in-one terminal/sniffer/monitor;
  • Binary-oriented logging engine;
  • Packet builder with convenient HEX editor and C-compatible packet templates;
  • Universal redirector allowing you to “redirect anything to anything”.

IO Ninja was born as an answer to our own needs, and I am confident that it will satisfy yours. Download the latest version of I/O Ninja here and take it for a spin — it is completely free for any non-commercial use.

Originally published at www.apsense.com

Evolve your approach to Application Performance Monitoring by adopting five best practices that are outlined and explored in this e-book, brought to you in partnership with BMC.

Topics:
iot protocols ,terminal

Published at DZone with permission of Vladimir Gladkov. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}