[TIL] How to Convert Coloured HTML Index to Linux Command

rootsh3ll-til-how-to-convert-coloured-html-index-to-linux-command

Have you ever wondered of making a coloured HTML index into a Linux Terminal command ? may be not but I did wonder how to do so.

and here I am with the complete step-by-step tutorial on how you can do the same, if ever thought to 😉

Checkout my new store for Best WiFi adapters for Hacking, Best-selling Pentesting Books and Best WiFi Boosters: Rootsh3ll rStore

Why would anyone do so ?

Well, this might be silly to many people but if you are familiar with my WiFi Hacking Series you know that I have an index which I have to follow as per article I post. So it is very much inappropriate for me to connect to the Internet, Log in to the browser and open my series’ index page and have a look at what I have done so far and get ready for another full-length article.

Pretty much slow isn’t it ?

So I used to use sticky notes app on my Windows 10 Desktop where I had dumped the whole index in a beautiful order(Just like on website) and just logging into the system I have my sticky notes opened. Too good!

I started noticing that over heating of my laptop causing me a lot of discomfort and reducing my productivity, which is already not too high.

Felt irritated, decided to go back to install Ubuntu with Bumblebee, which will not only keep the system way too cool but also give me extra backup than current system configuration. not to mention the nice wobbly effect(makes me feel good and comfortable)

Installed Ubuntu had all the fixes and now ready to write. But hey! I still need to check my status. checking the index is still a Lame idea!

There’s one notes app in Ubuntu Software Center, but that’s way too Ugly. I am not going with it.

Terminal has always been my good friend and I love it, always. Wouldn’t it be a better idea to dump my index into my Ubuntu system and just run it like a command any time I want to ? Yes! it is a great idea to me.

So I started doing some small level research on this. Didn’t found anything right away but knew what steps it would Involve.

Here is the breakdown that you can also think of easily:

  1. Dump the webpage on local directory, Obvious!
  2. Filter the output to reduce the clutter
  3. copy desired content in a shell script
  4. copy to /usr/bin and chmod +x
  5. execute the command

Above is a rough estimation of steps that would involve the procedure from HTML page to Linux command.

But there were some small issues involved also that you will see along.

For better understanding and learning I would like to encourage you to perform the tutorial along you are reading it. No need to install Ubuntu, you can do it on any Linux based OS 🙂

Step 1: Dump the webpage on local directory

Of course, in order to make a piece of content a command we do need to download it in first place then do filters,modifications etc on it.

I stumbled upon internet on how to download exact HTML so that I won’t lose the colour codes assigned with separate lines. You can see it in the index here.

quickly I got a result with this command.

  • elinks -dump -dump-color-mode 1 <URL>

which tells elinks to -dump the page in text format with color code 1.

In my case command goes like this, you too Try it along.

this is the output I got:

elinks-output

Pretty ugly, right ? ….very much!

This leads us to step #2 i.e Filter the output, reduce the clutter. But need to save the output in a text file first.

I did it with the simplest of all redirection operator, built-in to the shell. Here is what final command looked like:

a.txt is the first output filename.

Now we have saved the output in the text file. You can now unplug your Internet.

Moving on to Step #2

Step 2: Filter the elinks/HTML output to reduce the clutter

How are we going to do that ? I thought wouldn’t it be simple enough if I just simply crop the index part that I needed and save it to a new text file ? sounds nice to me. So I went to check the line number first to know which lines I need to crop first.

You can check the line number by using cat command:

Output now looked like this:

cat-n-line-numbers-output

better! You trying along or not ? Do it and learn along!

So I got that I need to to crop the text starting from line number 88 to 143, In my case for this output.

Again I stumbled upon Internet and got this result.

Here,

p – Print out the pattern space (to the standard output)

n – If auto-print is not disabled, print the pattern space, then, regardless, replace the pattern space with the next line of input. If there is no more input then sed exits without processing any more commands.

number Specifying a line number will match only that line in the input.

Pretty good! I got the filtered Index into a new text file a2.txt.

if you now cat the content of a2.txt, you will notice something weird.

Yes. What are those numbers in front of every coloured line ? those are the description by the elinks command that this is actually a link in the HTML format.

I don’t need that for sure. now need to filter that too.. :/

Okay! I covered this with 2 approaches

Approach 1:

Remove a specific numbers from a text using sed or awk command.

It didn’t go well for some reason and produced an incorrect output and didn’t remove the brackets. Have a try with this command that didn’t work:

Soon after which I decided to remove the brackets either along with the content and I encountered this result:

It worked for me and output(a3.txt) looked like this:

a3.txt

You also wouldn’t like to have these <- and Click to see notations in the output, or do you ? well, I didn’t like them as no links were associated with them neither I was supposed to click on any

So to remove them I updated the command I used previously into this:

If you notice closely, you will see the change is just within the single-quotes and nowhere else.

What I did is added a text “Click to see” and a string “<-” to be replaced with an empty string.

NOTE: Each string is told to sed by separating it by semi-colon ‘;’

It gave me exact output I was looking for, have a look:

a3.txt-final-output

Clean enough to move on to the next step! i.e

Step 3: Copy desired content in a shell script

If we simply make this file executable it wont give us any output. So it needs to be put in a script as a text and them be interpreted and printed as coloured Terminal output as it was doing earlier.

This is simple. Open any text editor and create a new file, I name the file rwsps.sh, Which the the course name by the way.

and typed into it using vi editor

First statement is called the HashBang, which tells the shell to use this specific interpreter for the script, BASH is used in our case.

second command i.e clear clears up the screen for better visibility.

Cat command take output from STDIN, denoted by ‘<<‘ till “__THIS__” string isn’t encountered again.

Means the moment cat command recieves “__THIS__” string in the text we will type afterwards, cat command will print the text and exit.

Now we need to append the a3.txt file which we filtered earlier with rwsps.sh, which is the newly made shell script.

We can append the content by simply using redirection operator.

NOTE: ‘>>’ will append the data to rwsps.sh, If you use ‘>’ it will over-wrtite the content of the file rwsps.sh with the contents of a3.txt. In other words it will delete rwsps.sh and make a copy of a3.txt with the name of rwsps.sh. So use wisely!

Now as we have provided rwsps.sh the content to print we will now add the delimiter to it which we told cat earlier i.e __THIS__

simply type:

and you are done.

Command is now ready. You need to copy it to /usr/bin directory in order to access it like a Terminal Command.

Step #4: Copy to /usr/bin and chmod +x

We will copy rwsps.sh in /usr/bin with name rwsps only. As Linux command needs to be executeble and they are independent of Extensions and also we want it to be short and simple 🙂

  1. sudo cp rwsps.sh /usr/bin/rwsps
  2. sudo chmod +x /usr/bin/rwsps

Now that the command is copied with SuperUser privilege and made Executable.

You can now execute command any time and anywhere.

Step 4: Execute the command

Open new Terminal and type:

  • rwsps

rwsps-terminal-command-output

This was the final output of my desire. Hope it will help you somehow, somewhere in your life.

I am happy now with my custom made command and also I learned something new today. So did you!

Do you have something new to make this process even simpler ? I would love to hear it right from you.

Share you views in the comments below and let me know How it helped you or How you can make it even better.

Index is now ready. See you in the next chapter of RWSPS.

Keep Learning

 

 

468
Shares
Share This
%d bloggers like this: