Search Repository

TECH.NE7.NL hosts a searchable repository of scripts and patterns I’ve created over the course of time. Mostly for my own usage but feel free to share or adapt any code or article from this site.

GNU/Linux BASH website probe using CURL

Small script I used to probe a website. Script should returns a message only when the state changes.

#!/bin/bash

function funcProbe {
  curl ${1} -k -I -s | grep "HTTP/1.1 200 OK" > /dev/null
  if [ $? -ne 0 ]; then
    if [ ! -f /tmp/prb-${2}-DOWN ]; then
      echo "[S2:Probe-${2}] Service DOWN Detected!"
    fi
    touch /tmp/prb-${2}-DOWN
    rm -f /tmp/prb-${2}-UP
  else
    if [ ! -f /tmp/prb-${2}-UP ]; then
      echo "[S2:Probe-${2}] Service UP Detected!"
    fi
    touch /tmp/prb-${2}-UP
    rm -f /tmp/prb-${2}-DOWN
  fi
}

funcProbe https://tech.ne7.nl/ Repository

GNU/Linux Python3 Mail Probe

A small script I used to quickly send a mail messages (probe).

#!/usr/bin/env python3
# Small mail probe...
from datetime import datetime
import smtplib
from email.message import EmailMessage
from email.mime.text import MIMEText

# Configured values...
senderid = "my-name"
subject = "MailProbe from " + senderid
body = "*** This is a mail probe message to verify the connection ***"
sender = "sender@example.com"
recipients = ["receiver@example.com"]
password = "store-password-somewhere-else"

# Mail sender function...
def send_email(subject, body, sender, recipients, password):
    now = datetime.now()
    msg = MIMEText(body)
    msg['X-Ne7-Probe-Firetime'] = now.strftime("%Y-%m-%dT%H:%M:%S")
    msg['Subject'] = subject
    msg['From'] = sender
    msg['To'] = ', '.join(recipients)
    smtp_server = smtplib.SMTP_SSL('smtp.example.com', 465)
    smtp_server.login(sender, password)
    smtp_server.sendmail(sender, recipients, msg.as_string())
    smtp_server.quit()

send_email(subject, body, sender, recipients, password)

GNU/Linux Just a quick way to Utilize a single query result value

Just a quick way to utilize a single query result value in a bash variable.

#!/bin/bash

# Just a bunch of dates relative to today...
TDAY=`date +%d`
MDAY=`date +%d -d "yesterday"`
TYMDAY=`date +%Y-%m-%d -d "yesterday"`
TMDAY=`date +%m-%d -d "yesterday"`
TY1DAY=`date +%Y-01-01 -d "yesterday"`
LYMDAY=`date +%Y-%m-%d -d "yesterday last year"`
LY1DAY=`date +%Y-01-01 -d "yesterday last year"`
YYEAR=`date +%Y -d "yesterday"`
LYEAR=`date +%Y -d "yesterday last year"`

# Get values from database...
Zl=$(mysql InfoRobot -u root --execute="SELECT COALESCE(ZOpbrengst, 0) AS ZOpbrengst from EDagCombi WHERE COALESCE(ZOpbrengst, -1) > -1 ORDER BY Jaar DESC, Maand DESC, Dag DESC LIMIT 1;" --silent)

DM=$(mysql InfoRobot -u root --execute="SELECT CONCAT(DAY(MAX(Tijdstip)), ' ', MONTHNAME(MAX(Tijdstip))) AS DayName from ZOpbrengst WHERE Tijdstip >  (now() - interval 2 day);" --silent)

ZM=$(mysql InfoRobot -u root --execute="SELECT ROUND(MaandTotaal) from ZMaandOpbrengst where Maand = (SELECT ThisMonth From CZThisMonth LIMIT 1);" --silent)

ZMLY=$(mysql InfoRobot -u root --execute="SELECT ROUND(MaandTotaal) from ZMaandOpbrengst where Maand = (SELECT ThisMonthLastYear From CZThisMonth LIMIT 1);" --silent)

Z1LY=$(mysql InfoRobot -u root --execute="SELECT ROUND(SUM(Opbrengst)) FROM ZOpbrengst WHERE Tijdstip >= '${LY1DAY}' and Tijdstip <= '${LYMDAY}';" --silent)

Z1TY=$(mysql InfoRobot -u root --execute="SELECT ROUND(SUM(Opbrengst)) FROM ZOpbrengst WHERE Tijdstip >= '${TY1DAY}' and Tijdstip <= '${TYMDAY}';" --silent)

P1TY=$(mysql InfoRobot -u root --execute="SELECT (MAX(T1afgenomen) - MIN(T1afgenomen)) + (MAX(T2afgenomen) - MIN(T2afgenomen)) FROM P1uitlezen WHERE date >= '${TY1DAY}' and date <= '${TYMDAY}';" --silent)

# Bash math functions using bc...
ZXPLY=$(echo "scale=2; (${ZM}/${ZMLY})" | bc)
ZZ=$(echo "scale=0; (${ZXPLY}*100)" | bc)
ZXYLY=$(echo "scale=2; ((${Z1TY}/${Z1LY})*100)" | bc)
FZ=$(echo "scale=0; (${ZXYLY})" | bc | xargs printf "%.*f" "0")

# Mail stuff...
/usr/local/sbin/send_to_mail.py SolarPower "${DM}: ${Zl} KWh, Total TY: ${Z1TY} KWh (${FZ}%)" > /dev/null

GNU/Linux PYTHON Script Read Values from JSON File

Another Python script to read values from a JSON file and store them in a mysql database.

import sys
import json
import csv
import MySQLdb
import mysql.connector
from datetime import datetime, date, timedelta

# Get yesterdays date...
yesterday = date.today() - timedelta(1)

# Create database connection...
dbconn = MySQLdb.connect(host='localhost', user='zonpan', passwd='maybe-store-pw-somwhere-else', db='databassename')
cursor = dbconn.cursor()

# Open Mysql Database...
with open('/path/to/directory/yesterday.json') as fd:
     json_data = json.load(fd)

# Insert Query...
add_zopbrengst = ("INSERT INTO ZOpbrengst (Tijdstip, Opbrengst) VALUES (%s, %s)")

# Parse JSON File and insert values...
datum = yesterday.strftime('%Y-%m-%d %H:%M')
opbrengst = json_data['data']['production']['yesterday'].replace(',', '.')
data_zo = (datum, opbrengst)
cursor.execute(add_zopbrengst, data_zo)
print(data_zo)

# Cleanup...
dbconn.commit()
cursor.close()
dbconn.close()

GNU/Linux PYTHON3 Script to Read HUE Sensor DATA

A small python3 script I wrote to read sensor data and store in a logg file.

#!/usr/bin/env python3
import sys
import json
import urllib.request
import requests
import parsedatetime as pdt

# Constants...
cal = pdt.Calendar()
hue_url_sensors = "http://path.to.hue.bridge/api/{api-key}/sensors/"
hue_url_lights = "http://path.to.hue.bridge/api/{api-key}/lights/"
air_url_1 = "http://path.to.other.sensor-1:8082/sensors/"
air_url_2 = "http://path.to.other.sensor-3:8082/sensors/"
rel_url = "http://path.to.relay.raspi:8081/relays/"
light_on = json.dumps({'on': True})
light_off = json.dumps({'on': False})


# Switch function...
def switch(key, value):
  action = 'N/A'
  url = hue_url_lights + key + "/state/"
  if value.lower() == 'off':
    result = requests.put(url, data=light_off.encode('utf-8'))
  elif value.lower() == 'on':
    result = requests.put(url, data=light_on.encode('utf-8'))
  print("INFO: ", "Light=", key, "Action=", value, "Result=", result)

# GetLights function
def getlights():
  # Retrieve huedata (lights)...
  output = ""
  web_data_l = urllib.request.urlopen(hue_url_lights)
  json_data_l = web_data_l.read()
  json_enc_l = web_data_l.info().get_content_charset('utf-8')
  json_lights = json.loads(json_data_l.decode(json_enc_l))
  for key in json_lights:
    if json_lights[key]['state']['reachable'] == True and json_lights[key]['state']['on'] == True:
      state = "on"
    else:
      state = "off"
    output = output + " " + str(key).zfill(2) + "\t" + json_lights[key]['name'].ljust(20) + str(key).ljust(2) + "\t" + state + "\n"
  return output

# GetSensors function
def getSensors():
  # Retreive Sensor Data
  output = ""
  web_data_l = urllib.request.urlopen(hue_url_sensors)
  json_data_l = web_data_l.read()
  json_enc_l = web_data_l.info().get_content_charset('utf-8')
  json_sensors = json.loads(json_data_l.decode(json_enc_l))
  for key in json_sensors:
    isValid = True
    if json_sensors[key]['type'] == 'ZLLLightLevel':
      value = json_sensors[key]['state']['lightlevel']
    elif json_sensors[key]['type'] == 'ZLLPresence':
      value = json_sensors[key]['state']['presence']
    elif json_sensors[key]['type'] == 'ZLLTemperature':
      value = json_sensors[key]['state']['temperature'] /100
    else:
      isValid = False
    if isValid == True:
      output = output + " " + str(key).zfill(2) + "\t"  + str(json_sensors[key]['name']).ljust(20) + "\t\t" + str(value) + ""
  return output

# GetSensors function
def getAirinfo(Url, DeviceId):
  # Retreive Sensor Data
  format = '%Y-%m-%dT%H:%M:%S'
  output = ""
  web_data_l = urllib.request.urlopen(Url)
  json_data_l = web_data_l.read()
  json_enc_l = web_data_l.info().get_content_charset('utf-8')
  json_sensors = json.loads(json_data_l.decode(json_enc_l))['sensors']
  for key in json_sensors:
    output = output + str(key) + ": " + str(json_sensors[key]['value']).ljust(5)
    datetxt = str(cal.parseDT(datetimeString=json_sensors[key]['LastModified'][5:-4])[0].strftime(format))
  return datetxt + " PS" + DeviceId + " " + output + "\n"

# GetRelayState function
def getRelayinfo(Url):
  # Retreive Sensor Data
  r1 = False
  r2 = False
  r8 = False
  VENT = "NA"
  output = ""
  web_data_l = urllib.request.urlopen(Url)
  json_data_l = web_data_l.read()
  json_enc_l = web_data_l.info().get_content_charset('utf-8')
  json_sensors = json.loads(json_data_l.decode(json_enc_l))['relays']
  for key in json_sensors:
    if key == '1':
      if json_sensors[key]['state'] == 1:
        r1=True
    elif key == '2':
      if json_sensors[key]['state'] == 1:
        r2=True
    elif key == '8':
      if json_sensors[key]['state'] == 1:
        r8=True
  if r1 == False and r2 == True:
    VENT = "HIGH"
  if r1 == False and r2 == False:
    VENT = "MEDIUM"
  if r1 == True:
    VENT = "LOW"
  output = " RE\tVENT".ljust(20) + "\t\t" + VENT
  return output


# Construct switch command...
with open("/var/log/ax.log", "a") as logfile:
  logfile.write(getAirinfo(air_url_1, "01"))
  logfile.write(getAirinfo(air_url_2, "02"))

GNU/Linux TMUX Quickref

tmux new -s {session-name}        -- create new session
tmux a (-t {session-name})        -- attach to default session or session-name
tmux ls                           -- list sessions
ctrl-b d                          -- detach session

ctrl-b c                          -- create new screen
ctrl-b [n|1-9]                    -- next screen | select screen
ctrl-b w                          -- select screen

ctrl-b "                          -- horizontal divider
ctrl-b %                          -- vertical divider
ctrl-b [up|down|left|right]       -- jump to screen with arrow keys
ctrl-b [pageup]                   -- scroll mode

Azure APIM Debugging

Follow the steps below to enable and retreive debug logging from a specific APIM resource.

First retrieve an access token. This can be done by de following PowerShell commands (cloud shell):

az login
az account set --subscription {subscription-id)
az account get-access-token

You can also use the folowing command to retrieve a list of apiIds so you can past the specific apiId path directly into the apiId key in the body.

az apim api list --resource-group {resource-group-name} --service-name {apim-service-name} --query "[].{Name:displayName, apiId:id}" -o table

Retrieve debug credentials by doing below HTTP POST.

POST https://management.azure.com/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.ApiManagement/service/{apim-service-name}/gateways/managed/listDebugCredentials?api-version=2023-05-01-preview HTTP/1.1
Content-Type: application/json
Authorization: Bearer {access-token}

{
    "credentialsExpireAfter": "PT1H",
    "apiId": "/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.ApiManagement/service/{apim-service-name}/apis/{api-name}",
    "purposes": ["tracing"]
}

Copy the token from the reponse and add it as a header value (Apim-Debug-Authorization) in the specific api-call where you want the debugging to take place…

GET https://example.com/rest/api HTTP/1.1
Apim-Debug-Authorization: aid=api-name...

In the header value of the response from the specific api there should also be a header value (Apim-Trace-Id) which you can use to retrieve the trace.

POST https://management.azure.com/subscriptions{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.ApiManagement/service/{apim-service-name}/gateways/managed/listTrace?api-version=2023-05-01-preview HTTP/1.1
Content-Type: application/json
Authorization: Bearer {access-token}

{ "traceId": "{apim-trace-id}" }

GNU/Linux UFW Quickref / Examples

sudo ufw status [numbered]

sudo ufw [delete] allow 443/tcp
sudo ufw [delete] allow from 10.0.0.1 proto tcp to any port 443 comment 'allow https trafic from 10.0.0.1'
sudo ufw [delete] [insert 1] reject from 10.0.0.0/24 comment 'Denies all trafic from specific subnet'

sudo ufw [delete] reject out to any proto tcp port 25
sudo ufw [delete] reject out to 192.168.5.0/24 proto tcp port 80,443

sudo ufw delete {rownumber}