Wilf

Networking and coding research

A place to dump my research, reference and developments.
It is quite possible that you have arrived here for my vintage Weight Watchers™ calculator page.

PHP Development

$_SERVER["http_referer"] not capturing the query string from search engines

I recently created a small project to capture the URL of sites that have referred to my own website.
Interestingly it seems that search engines have suppressed the query string part of the URL.

e.g. for the URL https://www.google.com/search?client=firefox-b-d&q=wilfs+corner I would expect to be able to capture this in its entirety using PHP's $_SERVER["http_referer"].
What is actually sent from the search engines is just the root domain. e.g. https://www.google.com/
Frustrating but then I suppose they have their own analytic tools.

Reverse IP lookup

As an extension to my 'referer' project I also capture the IP address of the client as they arrive at my site using $_SERVER["REMOTE_ADDR"].
I thought it would be an interesting exercise to see if I could do a reverse lookup on the IP address to identify any A (address) records held in each case.

I found a useful and free service at ipwho.is.
Using PHP's cURL is was able to provide any IP address and return some cool data. 

            
$id = $_POST["id"];
$ip = $_POST["ipaddress"];

$ch = curl_init('http://ipwho.is/'.$ip);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, false);
$ipwhois = json_decode(curl_exec($ch), true);
curl_close($ch);

$country = "";
$flag = "";
$city = "";
$domain = "";

if (isset($ipwhois['country']))
    $country = $ipwhois['country'];

if (isset($ipwhois['flag']['emoji']))
    $flag = $ipwhois['flag']['emoji'];

if (isset($ipwhois['city']))
    $city = $ipwhois['city'];

if (isset($ipwhois['connection']['domain']))
    $domain = $ipwhois['connection']['domain'];


echo "{ \"id\": ".$id.", \"city\": \"".$city."\", \"country\": \"".$country."\", \"flag\": \"".$flag."\", \"domain\": \"".$domain."\" }";
This was all done using an AJAX call where I used jQuery's .each() function on an arrary of IP's. The printed value is specific to my code requirement.
You can see the output by doing a default search on your own IP address: ipwho.is.
Neat tool. You can use it for free but there is a limit to the amount of lookups you can do each month.

Networking

Reference

  • Linux
    • Text editing: sudo nano <filename> - seems like a friendly editor (sudo allows for write permissions)
    • File viewing: cat <filename>
    • Locating a file: locate <filename>
    • grep - global regular expression print
      grep is a command-line utility for searcshing plain-text data sets for lines that match a regular expression. Its name comes from the ed command g/re/p, which has the same effect. grep was originally developed for the Unix operating system, but later available for all Unix-like systems and some others such as OS-9.
    • host
      • host <domain> 8.8.8.8 resolve <domain> using Google's public DNS at 8.8.8.8 - returns an IP address
      • nmap --dns-servers 8.8.4.4,8.8.8.8 -sL <ipaddress>/24 run a stealth scan on the returned IP address
  • Network Topology
    • Network topology is the arrangement of the elements (links, nodes, etc.) of a communication network. (Wikipedia)
  • NMAP (cheat sheet)
    • Output
      • -oN output in normal format
      • -oX output in XML format
      • -oG output in grepable format
    • Scan for machines on the network with no port scan
      • nmap -sn xxx.xxx.xxx.xxx
    • NSE - NMAP Scripting Engine
      • Verbose scan for vulnerabilities with the vuln script library while probing for service / version info (-v verbosity level)
        • nmap -sV -vvv --script=vuln xxx.xxx.xxx.xxx -p 443
    • Port status
      • nmap -sV -p 80,443 192.168.0.0/24 ports 80 and 443
      • nmap -sV -p 80,443 192.168.0.0/24 -open open ports only
      • nmap -sV -p 10-200 192.168.0.0/24 -open open ports in range 10 to 200
    • Enable OS detection, version detection, script scanning, and traceroute (-A aggressive scan)
      • nmap -A mydomain.com
    • Switches
      • -Pn Treat host as online. i.e. disable host discovery
      • -sU UDP scan (connectionless)
    • WAF - web application firewall detection (A Web Application Firewall (WAF) is specifically designed to protect websites from SQL injection, cross-site scripting, malformed HTTP packets, etc)
      • nmap -p 443 --script http-waf-detect --script-args="http-waf-detect.aggro,http-waf-detect.detectBodyChanges" <domain>
  • Network class
    • Class A - /8 - start address 0.0.0.0 - subnet mask 255.0.0.0 - CIDR notation /8 example: 10.80.1.72
    • Class B - /16 - start address 128.0.0.0 - subnet mask 255.255.0.0 - CIDR notation /16 example: 172.100.1.30
      • Note: 127.0.0.1 is reserved for loopback (localhost)
    • Class C - /24 - start address 192.0.0.0 - subnet mask 255.255.255.0 - CIDR notation /24 example: 192.168.1.15
  • Wireshark
    • MDNS - Multicast DNS
      In computer networking, the multicast DNS protocol resolves hostnames to IP addresses within small networks that do not include a local name server. It is a zero-configuration service, using essentially the same programming interfaces, packet formats and operating semantics as unicast Domain Name System.
    • ESP - Encapsulating Security Payload
      Encapsulating Security Payload (ESP) is a member of the Internet Protocol Security (IPsec) set of protocols that encrypt and authenticate the packets of data between computers using a Virtual Private Network (VPN). The focus and layer on which ESP operates makes it possible for VPNs to function securely.
    • IPsec - Internet Protocol Security
      In computing, Internet Protocol Security is a secure network protocol suite that authenticates and encrypts packets of data to provide secure encrypted communication between two computers over an Internet Protocol network. It is used in virtual private networks.
  • To research
TCP segment header
Offsets Octet 0 1 2 3
Octet Bit  7  6  5  4  3  2  1  0  7  6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0
0 0 Source port Destination port
4 32 Sequence number
8 64 Acknowledgment number (if ACK set)
12 96 Data offset Reserved
0 0 0
NS
CWR
ECE
URG
ACK
PSH
RST
SYN
FIN
 Window Size
16 128 Checksum Urgent pointer (if URG set)
20
 160
 Options (if data offset > 5. Padded at the end with "0" bits if necessary.)
⋮ ⋮
60 480

Blockchain

Researching Blockchain and everything Web3