Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

NCGREP, a Text Search Tool Based on ncurses

DZone's Guide to

NCGREP, a Text Search Tool Based on ncurses

A grep keyword search is often used to quickly locate files. But as you can imagine, when the search result set is large, it can be a lot of work.

· Big Data Zone ·
Free Resource

Hortonworks Sandbox for HDP and HDF is your chance to get started on learning, developing, testing and trying out new features. Each download comes preconfigured with interactive tutorials, sample data and developments from the Apache community.

NCGREP, which is based on the ncurses library to provide user interfaces, is a grep tool for searching text on a target directory. You can download the source code from GitHub here.

Background

As a VIM party, daily work development will often use a grep keyword search to quickly locate a file, as shown below:

Use grep for text search

Using grep for text search

However, there are two efficiency issues with this process:

  1. The result of the demo cannot be directly interacted with, and the path of the manually pasted file needs to be opened.
  2. The results are not grouped; the results are listed directly.

One can imagine that when the search result set is relatively large, it can be a lot of work.

That can be used for ag.vim's plug search, yeah? Yes, but this only solved the problem of interaction. This still does not solve the pain points of the result set grouping.

Use vim under ag for text search

Use vim under ag for text search

Ideas

There are great advantages in visualizing loading effects (lazy loading) when using a text-based global search with an IDE such as Eclipse.

Under the Eclipse global file search

Under the Eclipse global file search

Expect a Linux-based system to provide a similar search tool. Advantages (functions) are as follows:

  • The result set can interact directly.
  • The result set can be grouped.
  • The result set is loaded via lazy loading.

What is a class library based on a textual graphical interface? A general understanding of the next VIM, htop, and similar software based on a class library called ncurses can be achieved with practice.

Project

The project name is ncgrep. Why? Because of ngrep, egrep, and so on. (Note: ncgrep did not reference the grep source.)

Project Demo

ncgrep demo

Again, the code can be found here.

Main.cpp

Hide shrink copy code:

#include <ncurses.h>
#include <string>
#include <iostream>
#include <vector>
#include <cstdlib>
#include <unistd.h>
#include <thread>

#include "files.h"
#include "grep.h"
#include "tui.h"
#include "data.h"

using namespace std;

void init_screen();
void listen_keyboard();
void dispose_data();
void print_status_line(string msg);
unsigned long yMax, xMax, yWin, xWin;
unsigned long cur_line = 0;
long cur_dir_index = -1;
WINDOW * win;
vector<match_files> mfv;
vector<match_dirs> dirs, used_dirs;
char *dirname;
char *parttern;
int group_level;
bool do_moving = false;

int main(int argc, char ** argv)
{
    if (argc < 3) {
        cerr<<"Incorrect usage! ncgrep match_pattern file_path [search_group_level]"<<endl;
        return -1;
    }
    dirname = argv[2];
    parttern = argv[1];
    if (argc == 4) {
        group_level = atoi(argv[3]);
    } else {
        group_level = 1;
    }

    // Init screen
    init_screen();

    // Print window
    refresh_win(win, yWin, xWin, used_dirs, cur_dir_index, mfv, cur_line);

    // Dispose data
    thread sub_thread(dispose_data);

    // Keyboard input
    listen_keyboard();

    sub_thread.join();
    endwin();
    return 0;
}

void init_screen()
{
    // Ncurses initialization
    setlocale(LC_ALL,"");
    initscr();
    cbreak();
    noecho();
    keypad(stdscr, true);
    curs_set(0); // hiden the cursor
    getmaxyx(stdscr, yMax, xMax);
    yWin = long(yMax * 0.6);
    xWin = long(xMax * 0.8);
    win = newwin(yWin, xWin, (yMax - yWin) /2, (xMax - xWin) / 2);
    box(win, 0, 0);
    refresh();
}

void listen_keyboard() {
    int c;
    bool do_continue = true;
    while (do_continue && (c = getch())) {
        switch (c) {
            case 5: // ctrl-e
                if (cur_dir_index == -1) {
                    break;
                }
                cur_dir_index = -1;
                cur_line = 0;
                refresh_win(win, yWin, xWin, used_dirs, cur_dir_index, mfv, cur_line);
                break;
            case 10:
                if (cur_dir_index == -1) {
                    cur_dir_index = cur_line;
                    cur_line = used_dirs[cur_dir_index].start;
                    refresh_win(win, yWin, xWin, used_dirs, cur_dir_index, mfv, cur_line);
                } else {
                    string cmd = "vim " + mfv[cur_line].filename + " +" + to_string(mfv[cur_line].line);
                    system(cmd.c_str());
                    endwin();
                    init_screen();
                    refresh_win(win, yWin, xWin, used_dirs, cur_dir_index, mfv, cur_line);
                }
                break;
        }

        unsigned long min = 0;
        unsigned long max = used_dirs.size() == 0 ? 0 : used_dirs.size() - 1;
        if (cur_dir_index != -1) {
            min = used_dirs[cur_dir_index].start;
            max = used_dirs[cur_dir_index].start + used_dirs[cur_dir_index].length == 0 
                      ? 0 : used_dirs[cur_dir_index].start + used_dirs[cur_dir_index].length - 1;
        }
        switch (*keyname(c)) {
            case 'q':
                do_continue = false;
                break;
            case 'k':
                do_moving = true;
                if (cur_line == min) {
                    do_moving = false;
                    break;
                }
                refresh_win(win, yWin, xWin, used_dirs, cur_dir_index, mfv, --cur_line);
                do_moving = false;
                break;
            case 'j':
                do_moving = true;
                if (cur_line == max) {
                    do_moving = false;
                    break;
                }
                refresh_win(win, yWin, xWin, used_dirs, cur_dir_index, mfv, ++cur_line);
                do_moving = false;
                break;
            case 'o':
                if (cur_dir_index == -1) {
                    cur_dir_index = cur_line;
                    cur_line = used_dirs[cur_dir_index].start;
                    refresh_win(win, yWin, xWin, used_dirs, cur_dir_index, mfv, cur_line);
                } else {
                    string cmd = "vim " + mfv[cur_line].filename + " +" + to_string(mfv[cur_line].line);
                    system(cmd.c_str());
                    endwin();
                    init_screen();
                    refresh_win(win, yWin, xWin, used_dirs, cur_dir_index, mfv, cur_line);
                }
                break;
        }
    }
}

void dispose_data() {
        print_status_line("loadding...");
        vector<string> files_tmp;
        vector<match_files> mfv_tmp;
        try {
            dirs = getdirs(dirname, 0, group_level);
        } catch (runtime_error &e) {
            cerr<<e.what()<<endl;
            return;
        }
        unsigned long dirs_count = dirs.size();
        unsigned long files_count;
        // FOR GROUPs
        for (unsigned long i = 0; i < dirs_count; ++i) {
            files_tmp = listdir(dirs[i].dirname, group_level, dirs[i].mode);
            // FOR FILEs
            files_count = files_tmp.size();
            dirs[i].start = mfv.size();
            for (unsigned long j = 0; j < files_count; ++j) {
                try {
                    mfv_tmp = match_pattern(files_tmp[j], parttern);
                    mfv.insert(mfv.end(), mfv_tmp.begin(), mfv_tmp.end());
                } catch (runtime_error &e) {
                    continue;
                }
            }
            dirs[i].length = mfv.size() - dirs[i].start;
            if (dirs[i].length > 0) {
                match_dirs md;
                md.dirname = dirs[i].dirname;
                md.start = dirs[i].start;
                md.length = dirs[i].length;
                used_dirs.push_back(md);
            }
            // ONE GROUP RESULTS
            if (cur_dir_index == -1) {
                while (do_moving == true) {
                    std::this_thread::sleep_for(std::chrono::milliseconds(50));
                }
                refresh_win(win, yWin, xWin, used_dirs, cur_dir_index, mfv, cur_line);
                //std::this_thread::sleep_for(std::chrono::milliseconds(100));
            }
        }
        print_status_line("loaded");

    return;
}

void print_status_line(string msg) {
    mvprintw(yMax - 1, 0, msg.c_str());
    refresh();
    return;
}

Resources

Hortonworks Community Connection (HCC) is an online collaboration destination for developers, DevOps, customers and partners to get answers to questions, collaborate on technical articles and share code examples from GitHub.  Join the discussion.

Topics:
big data ,tutorial ,grep ,data analytics ,text search ,ncgrep ,ncurses

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}