GNU/Linux

All posts containing GNU/Linux-related scripts and patterns.

GNU/Linux BASH website probe using CURL

Small script I used to probe a website. Script should returns a message only when the state changes.

#!/bin/bash

function funcProbe {
  curl ${1} -k -I -s | grep "HTTP/1.1 200 OK" > /dev/null
  if [ $? -ne 0 ]; then
    if [ ! -f /tmp/prb-${2}-DOWN ]; then
      echo "[S2:Probe-${2}] Service DOWN Detected!"
    fi
    touch /tmp/prb-${2}-DOWN
    rm -f /tmp/prb-${2}-UP
  else
    if [ ! -f /tmp/prb-${2}-UP ]; then
      echo "[S2:Probe-${2}] Service UP Detected!"
    fi
    touch /tmp/prb-${2}-UP
    rm -f /tmp/prb-${2}-DOWN
  fi
}

funcProbe https://tech.ne7.nl/ Repository

GNU/Linux Python3 Mail Probe

A small script I used to quickly send a mail messages (probe).

#!/usr/bin/env python3
# Small mail probe...
from datetime import datetime
import smtplib
from email.message import EmailMessage
from email.mime.text import MIMEText

# Configured values...
senderid = "my-name"
subject = "MailProbe from " + senderid
body = "*** This is a mail probe message to verify the connection ***"
sender = "sender@example.com"
recipients = ["receiver@example.com"]
password = "store-password-somewhere-else"

# Mail sender function...
def send_email(subject, body, sender, recipients, password):
    now = datetime.now()
    msg = MIMEText(body)
    msg['X-Ne7-Probe-Firetime'] = now.strftime("%Y-%m-%dT%H:%M:%S")
    msg['Subject'] = subject
    msg['From'] = sender
    msg['To'] = ', '.join(recipients)
    smtp_server = smtplib.SMTP_SSL('smtp.example.com', 465)
    smtp_server.login(sender, password)
    smtp_server.sendmail(sender, recipients, msg.as_string())
    smtp_server.quit()

send_email(subject, body, sender, recipients, password)

GNU/Linux Just a quick way to Utilize a single query result value

Just a quick way to utilize a single query result value in a bash variable.

#!/bin/bash

# Just a bunch of dates relative to today...
TDAY=`date +%d`
MDAY=`date +%d -d "yesterday"`
TYMDAY=`date +%Y-%m-%d -d "yesterday"`
TMDAY=`date +%m-%d -d "yesterday"`
TY1DAY=`date +%Y-01-01 -d "yesterday"`
LYMDAY=`date +%Y-%m-%d -d "yesterday last year"`
LY1DAY=`date +%Y-01-01 -d "yesterday last year"`
YYEAR=`date +%Y -d "yesterday"`
LYEAR=`date +%Y -d "yesterday last year"`

# Get values from database...
Zl=$(mysql InfoRobot -u root --execute="SELECT COALESCE(ZOpbrengst, 0) AS ZOpbrengst from EDagCombi WHERE COALESCE(ZOpbrengst, -1) > -1 ORDER BY Jaar DESC, Maand DESC, Dag DESC LIMIT 1;" --silent)

DM=$(mysql InfoRobot -u root --execute="SELECT CONCAT(DAY(MAX(Tijdstip)), ' ', MONTHNAME(MAX(Tijdstip))) AS DayName from ZOpbrengst WHERE Tijdstip >  (now() - interval 2 day);" --silent)

ZM=$(mysql InfoRobot -u root --execute="SELECT ROUND(MaandTotaal) from ZMaandOpbrengst where Maand = (SELECT ThisMonth From CZThisMonth LIMIT 1);" --silent)

ZMLY=$(mysql InfoRobot -u root --execute="SELECT ROUND(MaandTotaal) from ZMaandOpbrengst where Maand = (SELECT ThisMonthLastYear From CZThisMonth LIMIT 1);" --silent)

Z1LY=$(mysql InfoRobot -u root --execute="SELECT ROUND(SUM(Opbrengst)) FROM ZOpbrengst WHERE Tijdstip >= '${LY1DAY}' and Tijdstip <= '${LYMDAY}';" --silent)

Z1TY=$(mysql InfoRobot -u root --execute="SELECT ROUND(SUM(Opbrengst)) FROM ZOpbrengst WHERE Tijdstip >= '${TY1DAY}' and Tijdstip <= '${TYMDAY}';" --silent)

P1TY=$(mysql InfoRobot -u root --execute="SELECT (MAX(T1afgenomen) - MIN(T1afgenomen)) + (MAX(T2afgenomen) - MIN(T2afgenomen)) FROM P1uitlezen WHERE date >= '${TY1DAY}' and date <= '${TYMDAY}';" --silent)

# Bash math functions using bc...
ZXPLY=$(echo "scale=2; (${ZM}/${ZMLY})" | bc)
ZZ=$(echo "scale=0; (${ZXPLY}*100)" | bc)
ZXYLY=$(echo "scale=2; ((${Z1TY}/${Z1LY})*100)" | bc)
FZ=$(echo "scale=0; (${ZXYLY})" | bc | xargs printf "%.*f" "0")

# Mail stuff...
/usr/local/sbin/send_to_mail.py SolarPower "${DM}: ${Zl} KWh, Total TY: ${Z1TY} KWh (${FZ}%)" > /dev/null

GNU/Linux PYTHON Script Read Values from JSON File

Another Python script to read values from a JSON file and store them in a mysql database.

import sys
import json
import csv
import MySQLdb
import mysql.connector
from datetime import datetime, date, timedelta

# Get yesterdays date...
yesterday = date.today() - timedelta(1)

# Create database connection...
dbconn = MySQLdb.connect(host='localhost', user='zonpan', passwd='maybe-store-pw-somwhere-else', db='databassename')
cursor = dbconn.cursor()

# Open Mysql Database...
with open('/path/to/directory/yesterday.json') as fd:
     json_data = json.load(fd)

# Insert Query...
add_zopbrengst = ("INSERT INTO ZOpbrengst (Tijdstip, Opbrengst) VALUES (%s, %s)")

# Parse JSON File and insert values...
datum = yesterday.strftime('%Y-%m-%d %H:%M')
opbrengst = json_data['data']['production']['yesterday'].replace(',', '.')
data_zo = (datum, opbrengst)
cursor.execute(add_zopbrengst, data_zo)
print(data_zo)

# Cleanup...
dbconn.commit()
cursor.close()
dbconn.close()

GNU/Linux PYTHON3 Script to Read HUE Sensor DATA

A small python3 script I wrote to read sensor data and store in a logg file.

#!/usr/bin/env python3
import sys
import json
import urllib.request
import requests
import parsedatetime as pdt

# Constants...
cal = pdt.Calendar()
hue_url_sensors = "http://path.to.hue.bridge/api/{api-key}/sensors/"
hue_url_lights = "http://path.to.hue.bridge/api/{api-key}/lights/"
air_url_1 = "http://path.to.other.sensor-1:8082/sensors/"
air_url_2 = "http://path.to.other.sensor-3:8082/sensors/"
rel_url = "http://path.to.relay.raspi:8081/relays/"
light_on = json.dumps({'on': True})
light_off = json.dumps({'on': False})


# Switch function...
def switch(key, value):
  action = 'N/A'
  url = hue_url_lights + key + "/state/"
  if value.lower() == 'off':
    result = requests.put(url, data=light_off.encode('utf-8'))
  elif value.lower() == 'on':
    result = requests.put(url, data=light_on.encode('utf-8'))
  print("INFO: ", "Light=", key, "Action=", value, "Result=", result)

# GetLights function
def getlights():
  # Retrieve huedata (lights)...
  output = ""
  web_data_l = urllib.request.urlopen(hue_url_lights)
  json_data_l = web_data_l.read()
  json_enc_l = web_data_l.info().get_content_charset('utf-8')
  json_lights = json.loads(json_data_l.decode(json_enc_l))
  for key in json_lights:
    if json_lights[key]['state']['reachable'] == True and json_lights[key]['state']['on'] == True:
      state = "on"
    else:
      state = "off"
    output = output + " " + str(key).zfill(2) + "\t" + json_lights[key]['name'].ljust(20) + str(key).ljust(2) + "\t" + state + "\n"
  return output

# GetSensors function
def getSensors():
  # Retreive Sensor Data
  output = ""
  web_data_l = urllib.request.urlopen(hue_url_sensors)
  json_data_l = web_data_l.read()
  json_enc_l = web_data_l.info().get_content_charset('utf-8')
  json_sensors = json.loads(json_data_l.decode(json_enc_l))
  for key in json_sensors:
    isValid = True
    if json_sensors[key]['type'] == 'ZLLLightLevel':
      value = json_sensors[key]['state']['lightlevel']
    elif json_sensors[key]['type'] == 'ZLLPresence':
      value = json_sensors[key]['state']['presence']
    elif json_sensors[key]['type'] == 'ZLLTemperature':
      value = json_sensors[key]['state']['temperature'] /100
    else:
      isValid = False
    if isValid == True:
      output = output + " " + str(key).zfill(2) + "\t"  + str(json_sensors[key]['name']).ljust(20) + "\t\t" + str(value) + ""
  return output

# GetSensors function
def getAirinfo(Url, DeviceId):
  # Retreive Sensor Data
  format = '%Y-%m-%dT%H:%M:%S'
  output = ""
  web_data_l = urllib.request.urlopen(Url)
  json_data_l = web_data_l.read()
  json_enc_l = web_data_l.info().get_content_charset('utf-8')
  json_sensors = json.loads(json_data_l.decode(json_enc_l))['sensors']
  for key in json_sensors:
    output = output + str(key) + ": " + str(json_sensors[key]['value']).ljust(5)
    datetxt = str(cal.parseDT(datetimeString=json_sensors[key]['LastModified'][5:-4])[0].strftime(format))
  return datetxt + " PS" + DeviceId + " " + output + "\n"

# GetRelayState function
def getRelayinfo(Url):
  # Retreive Sensor Data
  r1 = False
  r2 = False
  r8 = False
  VENT = "NA"
  output = ""
  web_data_l = urllib.request.urlopen(Url)
  json_data_l = web_data_l.read()
  json_enc_l = web_data_l.info().get_content_charset('utf-8')
  json_sensors = json.loads(json_data_l.decode(json_enc_l))['relays']
  for key in json_sensors:
    if key == '1':
      if json_sensors[key]['state'] == 1:
        r1=True
    elif key == '2':
      if json_sensors[key]['state'] == 1:
        r2=True
    elif key == '8':
      if json_sensors[key]['state'] == 1:
        r8=True
  if r1 == False and r2 == True:
    VENT = "HIGH"
  if r1 == False and r2 == False:
    VENT = "MEDIUM"
  if r1 == True:
    VENT = "LOW"
  output = " RE\tVENT".ljust(20) + "\t\t" + VENT
  return output


# Construct switch command...
with open("/var/log/ax.log", "a") as logfile:
  logfile.write(getAirinfo(air_url_1, "01"))
  logfile.write(getAirinfo(air_url_2, "02"))

GNU/Linux TMUX Quickref

tmux new -s {session-name}        -- create new session
tmux a (-t {session-name})        -- attach to default session or session-name
tmux ls                           -- list sessions
ctrl-b d                          -- detach session

ctrl-b c                          -- create new screen
ctrl-b [n|1-9]                    -- next screen | select screen
ctrl-b w                          -- select screen

ctrl-b "                          -- horizontal divider
ctrl-b %                          -- vertical divider
ctrl-b [up|down|left|right]       -- jump to screen with arrow keys
ctrl-b [pageup]                   -- scroll mode

GNU/Linux UFW Quickref / Examples

sudo ufw status [numbered]

sudo ufw [delete] allow 443/tcp
sudo ufw [delete] allow from 10.0.0.1 proto tcp to any port 443 comment 'allow https trafic from 10.0.0.1'
sudo ufw [delete] [insert 1] reject from 10.0.0.0/24 comment 'Denies all trafic from specific subnet'

sudo ufw [delete] reject out to any proto tcp port 25
sudo ufw [delete] reject out to 192.168.5.0/24 proto tcp port 80,443

sudo ufw delete {rownumber}

GNU/Linux Tripwire Quickref

# Add essential proc subs: /proc/sys, /proc/cpuinfo, /proc/modules

twadmin -m P /etc/tripwire/twpol.txt
tripwire --init

tripwire --check [--email-report]

tripwire --update --twrfile /var/lib/tripwire/report/servername-YYYYMMDD-HHMMSS.twr

Update script with last report:

lastfilename=(`ls -Art  /var/lib/tripwire/report/ | tail -n 1`)
sudo tripwire --update --twrfile /var/lib/tripwire/report/${lastfilename}

GNU/Linux Move Data Using Rsync

I was doing some maintenance on my local NAS. I used the command below to effectively move data from one location to another without losing file attributes.

rsync -avzhP --remove-source-files /mnt/das-2T-1/source/ /mnt/das-2T-1/destination/ [--dry-run]

You can also temporarily cancel the move and when you start the command again it continues where it stopped.

I also did some internal replication using replication tasks on my TrueNAS device. This creates a snapshot of the data set to replicate to an empty dataset. The destination dataset is overwritten so this option cannot be used to merge datasets. If you want to merge datasets is best to use the rsync option mentioned above.