Inevitable Comparison: TCP vs UDP

We use Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) to transfer data over the internet.

TCP is the most commonly used protocol because it offers a lot of built-in features such as connection, error-checking and ordering. Also packet delivery is guaranteed.

UDP is also one of the most used protocol. While TCP offers a lot of features, UDP just provides packet throwing. There is no connection, error-checking, ordering etc.

Before talking about use cases, let’s look at their features.
Connection

  • TCP: Connection-oriented (persistent)
  • UDP: Connectionless (fresh air)

Reliability

  • TCP: Reliable (Ordered, Guaranteed)
  • UDP: Unreliable (Drop, Disordering possiblities)

Weight

  • TCP: Heavy (Background mechanisms)
  • UDP: Light (Simply throw packets)
    Transport

  • TCP: Stream (Continous, Ordered)

  • UDP: Datagram (Unrelated delivery)
    Flow Control

  • TCP: Windowing, Congestion Avoidance

  • UDP: Nothing
    Speed

  • TCP: Slow (Resending, Recovering, Error-checking etc.)

  • UDP: Fast (Nothing)

We use TCP for important data because it has reliable and persistent pipeline. For example HTTP (Web), FTP (File), SMTP (Email), SSH (Terminal), SQL (DB Queries) built top of TCP.
We use UDP for unimportant, temporal data because there is no consistent mechanism for reliability or persistance. For example games, VoIP services, media streaming, broadcasting built with UDP.

Choosing the right protocol depends on your needs. Most of developers use TCP because it does pretty much everything built-in also it’s easy as file i/o. My suggestion is use TCP for less frequent, more important data; use UDP for more frequent, less important data.

I tried to tell you basic differences between TCP and UDP protocols but there is one more thing to understand (where the magic begins!): They both developed on Internet Protocol (IP). TCP provides ‘connection’ but connection is an illusion! There is a three-way handshake for connection establishment. Simply, TCP is UDP with advanced features. There were some good developers, they implemented useful solutions for industry needs. Did you ever wanted to go deep into connection establishment, reliability mechanisms? Do you want to implement your own TCP-like protocol?

2017/3/11 posted in  Network

Differences between TLS 1.2 and TLS 1.3

The current version of TLS, TLS 1.2, was defined in RFC 5246 and has been in use for the past eight years by the majority of all web browsers. Companies such as Cloudflare are already making TLS 1.3 available to their customers.

With the release of TLS 1.3, there are promises of enhanced security and speed. But how exactly do the changes from TLS 1.2 to TLS 1.3 cause these improvements? The following is a list of differences between TLS 1.2 and 1.3 that shows how the improvements are achieved.

This protocol was defined in an Internet Draft in April of 2017. TLS 1.3 contains improved security and speed. The major differences include:

• The list of supported symmetric algorithms has been pruned of all legacy algorithms. The remaining algorithms all use Authenticated Encryption with Associated Data (AEAD) algorithms.

• A zero-RTT (0-RTT) mode was added, saving a round-trip at connection setup for some application data at the cost of certain security properties.

• All handshake messages after the ServerHello are now encrypted.

• Key derivation functions have been re-designed, with the HMAC-based Extract-and-Expand Key Derivation Function (HKDF) being used as a primitive.

• The handshake state machine has been restructured to be more consistent and remove superfluous messages.

• ECC is now in the base spec and includes new signature algorithms. Point format negotiation has been removed in favor of single point format for each curve.

• Compression, custom DHE groups, and DSA have been removed, RSA padding now uses PSS.

• TLS 1.2 version negotiation verification mechanism was deprecated in favor of a version list in an extension.

• Session resumption with and without server-side state and the PSK-based ciphersuites of earlier versions of TLS have been replaced by a single new PSK exchange.

In short, the major benefits of TLS 1.3 vs that of TLS 1.2 is faster speeds and improved security.

Speed Benefits of TLS 1.3

TLS and encrypted connections have always added a slight overhead when it comes to web performance. HTTP/2 definitely helped with this problem, but TLS 1.3 helps speed up encrypted connections even more. To put it simply, with TLS 1.2, two round-trips have been needed to complete the TLS handshake. With 1.3, it requires only one round-trip, which in turn cuts the encryption latency in half. This helps those encrypted connections feel just a little bit snappier than before.

tls1.3

Another advantage of is that in a sense, on sites you have previously visited, you can now send data on the first message to the server. This is called a “zero round trip.” (0-RTT). And yes, this also results in improved load time times.

Improved Security With TLS 1.3

A big problem with TLS 1.2 is that it’s often not configured properly it leaves websites vulnerable to attacks. TLS 1.3 now removes obsolete and insecure features from TLS 1.2, including the following:

  • SHA-1
  • RC4
  • DES
  • 3DES
  • AES-CBC
  • MD5
  • Arbitrary Diffie-Hellman groups — CVE-2016-0701
  • EXPORT-strength ciphers – Responsible for FREAK and LogJam

Because the protocol is in a sense more simplified, this make it less likely for administrators and developers to misconfigure the protocol.

2016/2/19 posted in  Network

Sending raw Ethernet packets from a specific interface in C on Linux

Lately I’ve been writing some code to send packets to a specific MAC address from a specific interface. I’m sure this will come in handy again so here is how it goes:

Includes

    #include <netinet/in.h>
    #include <sys/socket.h>
    #include <arpa/inet.h>
    #include <net/if.h>
    #include <netinet/ip.h>
    #include <netinet/udp.h>
    #include <netinet/ether.h>
    #include <linux/if_packet.h>

Open the raw socket:

    int sockfd;
    ...
    /* Open RAW socket to send on */
    if ((sockfd = socket(AF_PACKET, SOCK_RAW, IPPROTO_RAW)) == -1) {
        perror("socket");
    }

Get the index of the interface to send on:

    struct ifreq if_idx;
    ...
    memset(&if_idx, 0, sizeof(struct ifreq));
    strncpy(if_idx.ifr_name, "eth0", IFNAMSIZ-1);
    if (ioctl(sock, SIOCGIFINDEX, &if_idx) < 0)
        perror("SIOCGIFINDEX");  

Get the MAC address of the interface to send on:

    struct ifreq if_mac;
    ...
    memset(&if_mac, 0, sizeof(struct ifreq));
    strncpy(if_mac.ifr_name, "eth0", IFNAMSIZ-1);
    if (ioctl(sock, SIOCGIFHWADDR, &if_mac) < 0)
        perror("SIOCGIFHWADDR");

Get the IP address of the interface to send on:

    struct ifreq if_ip;
    ...
    memset(&if_ip, 0, sizeof(struct ifreq));
    strncpy(if_ip.ifr_name, "eth0", IFNAMSIZ-1);
    if (ioctl(sock, SIOCGIFADDR, &if_ip) < 0)
        perror("SIOCGIFADDR");

Construct the Ethernet header:

    int tx_len = 0;
    char sendbuf[1024];
    struct ether_header *eh = (struct ether_header *) sendbuf;
    ...
    memset(sendbuf, 0, 1024);
    /* Ethernet header */
    eh->ether_shost[0] = ((uint8_t *)&if_mac.ifr_hwaddr.sa_data)[0];
    eh->ether_shost[1] = ((uint8_t *)&if_mac.ifr_hwaddr.sa_data)[1];
    eh->ether_shost[2] = ((uint8_t *)&if_mac.ifr_hwaddr.sa_data)[2];
    eh->ether_shost[3] = ((uint8_t *)&if_mac.ifr_hwaddr.sa_data)[3];
    eh->ether_shost[4] = ((uint8_t *)&if_mac.ifr_hwaddr.sa_data)[4];
    eh->ether_shost[5] = ((uint8_t *)&if_mac.ifr_hwaddr.sa_data)[5];
    eh->ether_dhost[0] = MY_DEST_MAC0;
    eh->ether_dhost[1] = MY_DEST_MAC1;
    eh->ether_dhost[2] = MY_DEST_MAC2;
    eh->ether_dhost[3] = MY_DEST_MAC3;
    eh->ether_dhost[4] = MY_DEST_MAC4;
    eh->ether_dhost[5] = MY_DEST_MAC5;
    eh->ether_type = htons(ETH_P_IP);
    tx_len += sizeof(struct ether_header);  

Construct the IP header:

    struct iphdr *iph = (struct iphdr *) (sendbuf + sizeof(struct ether_header));
    ...
    /* IP Header */
    iph->ihl = 5;
    iph->version = 4;
    iph->tos = 16; // Low delay
    iph->id = htons(54321);
    iph->ttl = ttl; // hops
    iph->protocol = 17; // UDP
    /* Source IP address, can be spoofed */
    iph->saddr = inet_addr(inet_ntoa(((struct sockaddr_in *)&if_ip.ifr_addr)->sin_addr));
    // iph->saddr = inet_addr("192.168.0.112");
    /* Destination IP address */
    iph->daddr = inet_addr("192.168.0.111");
    tx_len += sizeof(struct iphdr);  

Construct the UDP header:

    struct udphdr *udph = (struct udphdr *) (sendbuf + sizeof(struct iphdr) + sizeof(struct ether_header));
    ...
    /* UDP Header */
    udph->source = htons(3423);
    udph->dest = htons(5342);
    udph->check = 0; // skip
    tx_len += sizeof(struct udphdr);  

Fill in UDP payload:

    /* Packet data */
    sendbuf[tx_len++] = 0xde;
    sendbuf[tx_len++] = 0xad;
    sendbuf[tx_len++] = 0xbe;
    sendbuf[tx_len++] = 0xef;  

Fill in remaining header info:

    unsigned short csum(unsigned short *buf, int nwords)
    {
        unsigned long sum;
        for(sum=0; nwords>0; nwords--)
            sum += *buf++;
        sum = (sum >> 16) + (sum &0xffff);
        sum += (sum >> 16);
        return (unsigned short)(~sum);
    }
    ...
    /* Length of UDP payload and header */
    udph->len = htons(tx_len - sizeof(struct ether_header) - sizeof(struct iphdr));
    /* Length of IP payload and header */
    iph->tot_len = htons(tx_len - sizeof(struct ether_header));
    /* Calculate IP checksum on completed header */
    iph->check = csum((unsigned short *)(sendbuf+sizeof(struct ether_header)), sizeof(struct iphdr)/2);  

Send the raw Ethernet packet:

    /* Destination address */
    struct sockaddr_ll socket_address;
    ...
    /* Index of the network device */
    socket_address.sll_ifindex = if_idx.ifr_ifindex;
    /* Address length*/
    socket_address.sll_halen = ETH_ALEN;
    /* Destination MAC */
    socket_address.sll_addr[0] = MY_DEST_MAC0;
    socket_address.sll_addr[1] = MY_DEST_MAC1;
    socket_address.sll_addr[2] = MY_DEST_MAC2;
    socket_address.sll_addr[3] = MY_DEST_MAC3;
    socket_address.sll_addr[4] = MY_DEST_MAC4;
    socket_address.sll_addr[5] = MY_DEST_MAC5;
    /* Send packet */
    if (sendto(sock, sendbuf, tx_len, 0, (struct sockaddr*)&socket_address, sizeof(struct sockaddr_ll)) < 0)
        printf("Send failed\n");  

Code available here

2015/12/19 posted in  Network

Drive Headless Chromium with Python3

Browser Automation

Before we dive into any code, let’s talk about what a headless browser is and why it’s useful. In short, headless browsers are web browsers without a graphical user interface (GUI) and are usually controlled programmatically or via a command-line interface.

One of the many use cases for headless browsers is automating usability testing or testing browser interactions. If you’re trying to check how a page may render in a different browser or confirm that page elements are present after a user initiates a certain workflow, using a headless browser can provide a lot of assistance. In addition to this, traditional web-oriented tasks like web scraping can be difficult to do if the content is rendered dynamically (say, via Javascript). Using a headless browser allows easy access to this content because the content is rendered exactly as it would be in a full browser.

Headless Chrome and Python

Going Headless!

Setup

Before we get started, we need to install Chromium and download the latest ChromeDriver.

Next, let’s make a folder that will contain all of our files:

mkdir going_headless
#Now we can move the ChromeDriver into the directory that we just made:
mv Downloads/chromedriver going_headless/

Since we are using Selenium with Python, it’s a good idea to make a Python virtual environment. I use virtualenv, so if you use another virtual environment manager, the commands may be different.

$ cd going_headless && virtualenv -p python3 env  
$ source env/bin/activate

The next thing we need to do is install Selenium. If you’re not familiar with Selenium, it’s a suite of tools that allows developers to programmatically drive web browsers. It has language bindings for Java, C#, Ruby, Javascript (Node), and Python. To install the Selenium package for Python, we can run the following:

pip3 install selenium  

Example

Now that we’ve gotten all of that out of the way, let’s get to the fun part. Our goal is to write a script that searches for my name “Hamples” on back.re, and checks that a recent article I wrote about Android security is listed in the results. If you’ve followed the instructions above, you can use the headless version of Chromium with Selenium like so:

import os  
from selenium import webdriver  
from selenium.webdriver.common.keys import Keys  
from selenium.webdriver.chrome.options import Options`  

`chrome_options = Options()  
chrome_options.add_argument("--headless")  
chrome_options.binary_location = '/Applications/Google Chromium.app/Contents/MacOS/Google Chromium'`    

`driver = webdriver.Chrome(executable_path=os.path.abspath(“chromedriver"),   chrome_options=chrome_options)  
driver.get("https://back.re")`  

`magnifying_glass = driver.find_element_by_id("js-open-icon")  
if magnifying_glass.is_displayed():  
  magnifying_glass.click()  
else:  
  menu_button = driver.find_element_by_css_selector(".menu-trigger.local")  
  menu_button.click()`  

`search_field = driver.find_element_by_id("site-search")  
search_field.clear()  
search_field.send_keys("Hamples")  
search_field.send_keys(Keys.RETURN)  
assert "Nothing..." in driver.page_source   driver.close()` 

Example Explained

The driver.get function will be used navigate to the specified URL.

driver.get("https://back.re")

The back.re website is responsive, so we have to handle different conditions. As a result, we check to see if the expected search button is displayed. If it isn’t, we click the menu button to enter our search term.

magnifying_glass = driver.find_element_by_id("js-open-icon")  
if magnifying_glass.is_displayed():  
  magnifying_glass.click()  
else:  
  menu_button = driver.find_element_by_css_selector(".menu-trigger.local")  
  menu_button.click()  

Now we clear the search field, search for my name, and send the RETURN key to the drive.

search_field = driver.find_element_by_id("site-search")  
search_field.clear()  
search_field.send_keys("Olabode")  
search_field.send_keys(Keys.RETURN)

We check to make sure that the blog post title from one of my most recent posts is in the page’s source.

assert "Nothing..." in driver.page_source

And finally, we close the browser.

driver.close().

Benchmarks

Head to Headless

So, it’s cool that we can now control Chrome using Selenium and Python without having to see a browser window, but we are more interested in the performance benefits we talked about earlier. Using the same script above, we profiled the time it took to complete the tasks, peak memory usage, and CPU percentage. We polled CPU and memory usage with psutil and measured the time for task completion using timeit.

For our small script, there were very small differences in the amount of time taken to complete the task (4.3%), memory usage (.5%), and CPU percentage (5.2%). While the gains in our example were very minimal, these gains would prove to be beneficial in a test suite with dozens of tests.

Resources

Chrome Links:

Selenium Links:

2015/6/19 posted in  Network