Author name: Tim van Kooten Niekerk

About me / Hey I’m Tim. I work as a systems integration specialist for a large educational institution in the Netherlands. A part of my free time I spend making and creating music.

GNU/Linux BASH website probe using CURL

Small script I used to probe a website. Script should returns a message only when the state changes.

#!/bin/bash

function funcProbe {
  curl ${1} -k -I -s | grep "HTTP/1.1 200 OK" > /dev/null
  if [ $? -ne 0 ]; then
    if [ ! -f /tmp/prb-${2}-DOWN ]; then
      echo "[S2:Probe-${2}] Service DOWN Detected!"
    fi
    touch /tmp/prb-${2}-DOWN
    rm -f /tmp/prb-${2}-UP
  else
    if [ ! -f /tmp/prb-${2}-UP ]; then
      echo "[S2:Probe-${2}] Service UP Detected!"
    fi
    touch /tmp/prb-${2}-UP
    rm -f /tmp/prb-${2}-DOWN
  fi
}

funcProbe https://tech.ne7.nl/ Repository

GNU/Linux Python3 Mail Probe

A small script I used to quickly send a mail messages (probe).

#!/usr/bin/env python3
# Small mail probe...
from datetime import datetime
import smtplib
from email.message import EmailMessage
from email.mime.text import MIMEText

# Configured values...
senderid = "my-name"
subject = "MailProbe from " + senderid
body = "*** This is a mail probe message to verify the connection ***"
sender = "sender@example.com"
recipients = ["receiver@example.com"]
password = "store-password-somewhere-else"

# Mail sender function...
def send_email(subject, body, sender, recipients, password):
    now = datetime.now()
    msg = MIMEText(body)
    msg['X-Ne7-Probe-Firetime'] = now.strftime("%Y-%m-%dT%H:%M:%S")
    msg['Subject'] = subject
    msg['From'] = sender
    msg['To'] = ', '.join(recipients)
    smtp_server = smtplib.SMTP_SSL('smtp.example.com', 465)
    smtp_server.login(sender, password)
    smtp_server.sendmail(sender, recipients, msg.as_string())
    smtp_server.quit()

send_email(subject, body, sender, recipients, password)

GNU/Linux Just a quick way to Utilize a single query result value

Just a quick way to utilize a single query result value in a bash variable.

#!/bin/bash

# Just a bunch of dates relative to today...
TDAY=`date +%d`
MDAY=`date +%d -d "yesterday"`
TYMDAY=`date +%Y-%m-%d -d "yesterday"`
TMDAY=`date +%m-%d -d "yesterday"`
TY1DAY=`date +%Y-01-01 -d "yesterday"`
LYMDAY=`date +%Y-%m-%d -d "yesterday last year"`
LY1DAY=`date +%Y-01-01 -d "yesterday last year"`
YYEAR=`date +%Y -d "yesterday"`
LYEAR=`date +%Y -d "yesterday last year"`

# Get values from database...
Zl=$(mysql InfoRobot -u root --execute="SELECT COALESCE(ZOpbrengst, 0) AS ZOpbrengst from EDagCombi WHERE COALESCE(ZOpbrengst, -1) > -1 ORDER BY Jaar DESC, Maand DESC, Dag DESC LIMIT 1;" --silent)

DM=$(mysql InfoRobot -u root --execute="SELECT CONCAT(DAY(MAX(Tijdstip)), ' ', MONTHNAME(MAX(Tijdstip))) AS DayName from ZOpbrengst WHERE Tijdstip >  (now() - interval 2 day);" --silent)

ZM=$(mysql InfoRobot -u root --execute="SELECT ROUND(MaandTotaal) from ZMaandOpbrengst where Maand = (SELECT ThisMonth From CZThisMonth LIMIT 1);" --silent)

ZMLY=$(mysql InfoRobot -u root --execute="SELECT ROUND(MaandTotaal) from ZMaandOpbrengst where Maand = (SELECT ThisMonthLastYear From CZThisMonth LIMIT 1);" --silent)

Z1LY=$(mysql InfoRobot -u root --execute="SELECT ROUND(SUM(Opbrengst)) FROM ZOpbrengst WHERE Tijdstip >= '${LY1DAY}' and Tijdstip <= '${LYMDAY}';" --silent)

Z1TY=$(mysql InfoRobot -u root --execute="SELECT ROUND(SUM(Opbrengst)) FROM ZOpbrengst WHERE Tijdstip >= '${TY1DAY}' and Tijdstip <= '${TYMDAY}';" --silent)

P1TY=$(mysql InfoRobot -u root --execute="SELECT (MAX(T1afgenomen) - MIN(T1afgenomen)) + (MAX(T2afgenomen) - MIN(T2afgenomen)) FROM P1uitlezen WHERE date >= '${TY1DAY}' and date <= '${TYMDAY}';" --silent)

# Bash math functions using bc...
ZXPLY=$(echo "scale=2; (${ZM}/${ZMLY})" | bc)
ZZ=$(echo "scale=0; (${ZXPLY}*100)" | bc)
ZXYLY=$(echo "scale=2; ((${Z1TY}/${Z1LY})*100)" | bc)
FZ=$(echo "scale=0; (${ZXYLY})" | bc | xargs printf "%.*f" "0")

# Mail stuff...
/usr/local/sbin/send_to_mail.py SolarPower "${DM}: ${Zl} KWh, Total TY: ${Z1TY} KWh (${FZ}%)" > /dev/null

GNU/Linux PYTHON Script Read Values from JSON File

Another Python script to read values from a JSON file and store them in a mysql database.

import sys
import json
import csv
import MySQLdb
import mysql.connector
from datetime import datetime, date, timedelta

# Get yesterdays date...
yesterday = date.today() - timedelta(1)

# Create database connection...
dbconn = MySQLdb.connect(host='localhost', user='zonpan', passwd='maybe-store-pw-somwhere-else', db='databassename')
cursor = dbconn.cursor()

# Open Mysql Database...
with open('/path/to/directory/yesterday.json') as fd:
     json_data = json.load(fd)

# Insert Query...
add_zopbrengst = ("INSERT INTO ZOpbrengst (Tijdstip, Opbrengst) VALUES (%s, %s)")

# Parse JSON File and insert values...
datum = yesterday.strftime('%Y-%m-%d %H:%M')
opbrengst = json_data['data']['production']['yesterday'].replace(',', '.')
data_zo = (datum, opbrengst)
cursor.execute(add_zopbrengst, data_zo)
print(data_zo)

# Cleanup...
dbconn.commit()
cursor.close()
dbconn.close()

GNU/Linux PYTHON3 Script to Read HUE Sensor DATA

A small python3 script I wrote to read sensor data and store in a logg file.

#!/usr/bin/env python3
import sys
import json
import urllib.request
import requests
import parsedatetime as pdt

# Constants...
cal = pdt.Calendar()
hue_url_sensors = "http://path.to.hue.bridge/api/{api-key}/sensors/"
hue_url_lights = "http://path.to.hue.bridge/api/{api-key}/lights/"
air_url_1 = "http://path.to.other.sensor-1:8082/sensors/"
air_url_2 = "http://path.to.other.sensor-3:8082/sensors/"
rel_url = "http://path.to.relay.raspi:8081/relays/"
light_on = json.dumps({'on': True})
light_off = json.dumps({'on': False})


# Switch function...
def switch(key, value):
  action = 'N/A'
  url = hue_url_lights + key + "/state/"
  if value.lower() == 'off':
    result = requests.put(url, data=light_off.encode('utf-8'))
  elif value.lower() == 'on':
    result = requests.put(url, data=light_on.encode('utf-8'))
  print("INFO: ", "Light=", key, "Action=", value, "Result=", result)

# GetLights function
def getlights():
  # Retrieve huedata (lights)...
  output = ""
  web_data_l = urllib.request.urlopen(hue_url_lights)
  json_data_l = web_data_l.read()
  json_enc_l = web_data_l.info().get_content_charset('utf-8')
  json_lights = json.loads(json_data_l.decode(json_enc_l))
  for key in json_lights:
    if json_lights[key]['state']['reachable'] == True and json_lights[key]['state']['on'] == True:
      state = "on"
    else:
      state = "off"
    output = output + " " + str(key).zfill(2) + "\t" + json_lights[key]['name'].ljust(20) + str(key).ljust(2) + "\t" + state + "\n"
  return output

# GetSensors function
def getSensors():
  # Retreive Sensor Data
  output = ""
  web_data_l = urllib.request.urlopen(hue_url_sensors)
  json_data_l = web_data_l.read()
  json_enc_l = web_data_l.info().get_content_charset('utf-8')
  json_sensors = json.loads(json_data_l.decode(json_enc_l))
  for key in json_sensors:
    isValid = True
    if json_sensors[key]['type'] == 'ZLLLightLevel':
      value = json_sensors[key]['state']['lightlevel']
    elif json_sensors[key]['type'] == 'ZLLPresence':
      value = json_sensors[key]['state']['presence']
    elif json_sensors[key]['type'] == 'ZLLTemperature':
      value = json_sensors[key]['state']['temperature'] /100
    else:
      isValid = False
    if isValid == True:
      output = output + " " + str(key).zfill(2) + "\t"  + str(json_sensors[key]['name']).ljust(20) + "\t\t" + str(value) + ""
  return output

# GetSensors function
def getAirinfo(Url, DeviceId):
  # Retreive Sensor Data
  format = '%Y-%m-%dT%H:%M:%S'
  output = ""
  web_data_l = urllib.request.urlopen(Url)
  json_data_l = web_data_l.read()
  json_enc_l = web_data_l.info().get_content_charset('utf-8')
  json_sensors = json.loads(json_data_l.decode(json_enc_l))['sensors']
  for key in json_sensors:
    output = output + str(key) + ": " + str(json_sensors[key]['value']).ljust(5)
    datetxt = str(cal.parseDT(datetimeString=json_sensors[key]['LastModified'][5:-4])[0].strftime(format))
  return datetxt + " PS" + DeviceId + " " + output + "\n"

# GetRelayState function
def getRelayinfo(Url):
  # Retreive Sensor Data
  r1 = False
  r2 = False
  r8 = False
  VENT = "NA"
  output = ""
  web_data_l = urllib.request.urlopen(Url)
  json_data_l = web_data_l.read()
  json_enc_l = web_data_l.info().get_content_charset('utf-8')
  json_sensors = json.loads(json_data_l.decode(json_enc_l))['relays']
  for key in json_sensors:
    if key == '1':
      if json_sensors[key]['state'] == 1:
        r1=True
    elif key == '2':
      if json_sensors[key]['state'] == 1:
        r2=True
    elif key == '8':
      if json_sensors[key]['state'] == 1:
        r8=True
  if r1 == False and r2 == True:
    VENT = "HIGH"
  if r1 == False and r2 == False:
    VENT = "MEDIUM"
  if r1 == True:
    VENT = "LOW"
  output = " RE\tVENT".ljust(20) + "\t\t" + VENT
  return output


# Construct switch command...
with open("/var/log/ax.log", "a") as logfile:
  logfile.write(getAirinfo(air_url_1, "01"))
  logfile.write(getAirinfo(air_url_2, "02"))

GNU/Linux TMUX Quickref

tmux new -s {session-name}        -- create new session
tmux a (-t {session-name})        -- attach to default session or session-name
tmux ls                           -- list sessions
ctrl-b d                          -- detach session

ctrl-b c                          -- create new screen
ctrl-b [n|1-9]                    -- next screen | select screen
ctrl-b w                          -- select screen

ctrl-b "                          -- horizontal divider
ctrl-b %                          -- vertical divider
ctrl-b [up|down|left|right]       -- jump to screen with arrow keys
ctrl-b [pageup]                   -- scroll mode

Azure APIM Debugging

Follow the steps below to enable and retreive debug logging from a specific APIM resource.

First retrieve an access token. This can be done by de following PowerShell commands (cloud shell):

az login
az account set --subscription {subscription-id)
az account get-access-token

You can also use the folowing command to retrieve a list of apiIds so you can past the specific apiId path directly into the apiId key in the body.

az apim api list --resource-group {resource-group-name} --service-name {apim-service-name} --query "[].{Name:displayName, apiId:id}" -o table

Retrieve debug credentials by doing below HTTP POST.

POST https://management.azure.com/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.ApiManagement/service/{apim-service-name}/gateways/managed/listDebugCredentials?api-version=2023-05-01-preview HTTP/1.1
Content-Type: application/json
Authorization: Bearer {access-token}

{
    "credentialsExpireAfter": "PT1H",
    "apiId": "/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.ApiManagement/service/{apim-service-name}/apis/{api-name}",
    "purposes": ["tracing"]
}

Copy the token from the reponse and add it as a header value (Apim-Debug-Authorization) in the specific api-call where you want the debugging to take place…

GET https://example.com/rest/api HTTP/1.1
Apim-Debug-Authorization: aid=api-name...

In the header value of the response from the specific api there should also be a header value (Apim-Trace-Id) which you can use to retrieve the trace.

POST https://management.azure.com/subscriptions{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.ApiManagement/service/{apim-service-name}/gateways/managed/listTrace?api-version=2023-05-01-preview HTTP/1.1
Content-Type: application/json
Authorization: Bearer {access-token}

{ "traceId": "{apim-trace-id}" }

GNU/Linux UFW Quickref / Examples

sudo ufw status [numbered]

sudo ufw [delete] allow 443/tcp
sudo ufw [delete] allow from 10.0.0.1 proto tcp to any port 443 comment 'allow https trafic from 10.0.0.1'
sudo ufw [delete] [insert 1] reject from 10.0.0.0/24 comment 'Denies all trafic from specific subnet'

sudo ufw [delete] reject out to any proto tcp port 25
sudo ufw [delete] reject out to 192.168.5.0/24 proto tcp port 80,443

sudo ufw delete {rownumber}