Jenkins Should Be A Robot

Random thought of the day.

The Jenkins icon depicts a human butler, the suggestion being we are employing another human to go away and do some work for us.

To me, Jenkins is – whole or at least in part – the Machine-that-makes-the-machine.

When combined with an automated API-based provisioning engine, using clouds such as AWS, we have all the digital machinery we need to create the end-machine that will serve the end-consumer.

Remeber, Jenkins makes the digital software products that go on to serve end-consumers; it isn’t an end-product iteslf.

To re-interpret an old phrase: Jenkins is the means to the end, not the end itself.

The analogy I would make is: Jenkins is like a robot-arm (or die-cast machine) in a car factory. It is part of the machinery that builds the car, but it is not part of the car itself.

Car factories are industrialised. Typically they are designed to mass-manufacture at scale. They can stamp out millions of cars.

My argument is, essentially, that robot-arm is to car what Jenkins is to application environment.

Coming back to the original point of this article, Jenkins is a machine that can mass-manufacture software artifacts, not a human that hand-makes each one at low scale.

Thus, at least by this logic, Jenkins should be a robot, not a human. 🙂

An Essay on Capital Economics

Let’s take a detour from technology for a short while…

I have often tried to concider how the money supply grows.

It stands to reason that when the amount of “stuff” (i.e. production output) changes then the money supply must also change in lock-step, in order to avoid any destabilisation from significant & unwanted amounts of inflation or deflation. Total money supply is in essence an abstract (technological) representation of total “stuff” supply (i.e. products, goods, and services) that make up the surplus by which Economics exists to lay out a just mechanism for distribution to the populace at large.

Concider the main actors in the economic system: 1) Soverign, 2) Government, 3) Central Banks, 4) Retail Banks, 5) Businesses (providers) & individuals (consumers).

The authorities (Soverign, Government, Central Bank) say to their Retail Banks:

You can have the power to create brand new money out of thin air in the form of loans, however you must be able to attract that money back – with interest – to show that the recipient of the loan created new value for the economy (i.e. the societal surplus). This way, money supply should grow in-line with production output.

And remember, each currency unit added to the money supply will reside in bank accounts, and so, due to fractional reserve banking, will be multiplied many times over (to account for screwges in the system that hoard wealth). Thus, demand is amplified and circling back to the original loan, this was to increase supply; they both were able to grow together.

The ultimate purpose of this capitalistic economic mechanism, or system, is to move society away from scarcity and towards abundance.

But what of the profit motive? This is simply a means to utilize the greed of the individual in a way that helps rather than harms the society around them. The deal is: individuals can have more of the surplus of society but in return they must also be adding back into the surplus in a net positive way, by putting in more extra value than they are taking out by consuming stuff (products & services). Therein lies the genius of capitalisim; aligning personal greed with societal improvement.

If, for a period of time, supply has not kept up with demand (e.g. a supply chain shock), then here enters a second way to increase money supply. The Central Bank is empowered by the Government/Sovereign to offer increased interest rates to banks who store their on-hand deposits overnight with them. If Central Bank base rates are high enough then banks will elect to keep their money with the Central Bank rather than using it to tether a higher dept to equity ratio (remember banks can only be leveraged on their deposits by up to a certain ratio, usually by law). The net effect is banks lend less to the public, but also earn interest which is actually a net addition to the money supply.

Finally, we have Quantitative Easing (QE) & Quantitative Tightening (QT). This is another way to increase money supply but with the notable difference that an a account, or balance, is kept of exactly how much money – usually in the form of government or company bond purchases – has been added into the system. QE has the effect of stimulating demand by increasing liquidity throughout the system. This includes demand for assets such as property and securities (stocks). QT has the opposite effect; liquidity is pulled out of the system thereby decreasing demand.

In conclusion, balance has to be maintained between supply and demand forces whilst, over time, the economy is grown. This is how we achieve prosperity through abundance for all. And – rather excitingly for technologists – the author believes technology plays an every increasing role in this beautiful process as society becomes more and more advanced.

Perspectives on DevOps #1

Short, succinct thoughts on the DevOps movement.

DevOps is about accelerating customer value delivery.

Some principles that can help enable this software mission:

Balance planning & doing – both are needed, but in the right amount

Ship often – small incremental changes

Ship reliably – have a stable build machine/pipeline

Fail fast, rollback safely – changes can be quickly & easily reversed

Always listenting – responding continuously to feedback

Always improving – have a product roadmap

Also, a short comment on people teams/squads based upon statements I listened to recently but were recorded a long time ago from Elon Musk:

The future is probabilistic.

A team of people is a bunch of probabilities pointing in some direction.

Aligning those probabilities to the same direction will maximise the chance of a successful outcome & future.

AWS REST API Gateway #1

My journey to a basic achievement: “Hello World”

As a learning stepping-stone for one of my recent projects, I decided to set myself the challenge of getting a simple AWS REST API Gateway up and running. In short, this service was to be implemented using Infrastructure-as-Code (IaC) to create a functional REST API Gateway in AWS that will return a message of “Hello World” on the API’s root public URL.

The simplest REST API integration is called ‘MockIntegration‘. It does not require a separate backend, such as Lambda function or http endpoint, and so I chose to use this whilst getting started.

The chosen IaC language/library was AWS Cloud Development Kit (CDK) for TypeScript.

Along The Way

I learned that it can sometimes be fairly straight-forward to create the items/objects you want within the graphical AWS web console, but it can be more challenging to then replicate what you have built using IaC.

The main example of this was figuring out where to put the actual “Hello World” message in CDK code. I knew where to do this in the AWS console (under the GET method’s integration response) but was struggling to translate this knowledge into code.

In the end it was the aws command line tool that came to my rescue, giving me the responseTemplate syntax that I needed. The following command revealed how to do it (after I had added the integration response in the AWS console first, and re-deployed the API stage so the config was live):

> aws apigateway get-integration-response --rest-api-id <api_id> --resource-id <resource_id> --http-method GET --status-code 200

The Code

Anyhow, the code below assumes you have already run cdk init, to get the basic app/stack structure, and are inserting code into a stack constructor method.

First, I created the main API instance:

readonly api: RestApi

this.api = new RestApi(this, 'MyRestApi', {
   description: 'My REST API Gateway',
   endpointTypes: [aws_apigateway.EndpointType.REGIONAL],
   deployOptions: {
      stageName: 'development',
   }
})

Next, the root MockIntegration is needed:

this.api.root.addMethod('GET', new MockIntegration({
   integrationResponses: [
      {
         statusCode: '200',
         responseTemplates: {
            "application/json": "#set($inputRoot = $input.path('$'))\n{ message : \"Hello World\" }"
         }
      },
   ],
   passthroughBehaviour: PassthroughBehaviour.NEVER,
   requestTemplates: {
      'application/json': '{ "statusCode": 200 }',
   },
}), {
   methodResponses: [
      {
         statusCode: '200',
         responseModels: {
            'application/json': Model.EMPTY_MODEL
         }
      },
   ],
})

The Result

Finally, with the above code blocks in my stack constructor method, I had gathered together a sufficient amount of instruction for AWS to deploy my stack for me:

> npx cdk deploy <stack_name> [--profile <profile_name>]

Once successfully deployed, browsing to the output URL provided the desired “Hello World” message which I had been seeking.

Goal achieved! Overall, as a learner of CDK for TypeScript, I found this to be a useful exercise to understand AWS API Gateway better.

Useful Links

Here are some links that helped me on my journey:

Official AWS CDK Docs

https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_apigateway-readme.html

https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_apigateway.MockIntegration.html

Other

https://www.alexdebrie.com/posts/api-gateway-elements

iPhone Playback Device Selector

Using Apple Shortcuts.

I am a recent convert to Apple. It started with a personal Macbook Air, but now I am well and truely into the Apple eco-system having a bunch of their hardware.

When I swapped from Samsung/Android over to iPhone, one thing that seemed missing by default (I am willing to be corrected if wrong here) but is really useful is the ability to easily switch between playback devices, such as the iPhone itself, bluetooth headphones, and bluetooth speakers.

A cool way I have found to achieve this from my home app screen is to use the Shortcuts app/feature that you get with iOS to create your own macro-like runbooks.

Here is how I did it…

Create A Shortcut for each Device

  1. Opent the Shortcuts app
  2. Click the Plus (+) button at the top right to add a new shortcut
  3. In the Search for apps and actions box, type “Playback”, then select the Scripting action for Change Playback Destination
  4. Choose a playback device from the list
  5. Give your new Shortcut a name at the top of the screen
  6. Press the Cross (x) button at the top right of the screen to save the new shortcut
  7. This should take you back to the All Shortcuts screen, where you should see your newly created shortcut
  8. Repeat for each one of your playback devices

Create A Menu for the device shortcuts, Add to home screen

Next we create a menu linked to an app icon which we can place on our home screen.

  1. As we did before, in the Shortcuts app, click the Plus (+) button at the top right to add a new shortcut
  2. This time, search for Choose from menu in the search box and add it
  3. Give your new menu a promt, something like “Select Device”
  4. Add a menu item for each of your playback devices and give them an intuative name (so you know which option is for which device)
  5. Then, for each playback device, search for and add a Run shortcut action, place the action under the respective menu item, and link the action to the relevant device shortcut (created earlier)
  6. Now you should have a viable menu structure, with options that each set a different playback device
  7. The final step is to create an icon to run your new menu from the home screen. Do this by pressing the options/preference/sliders button (not sure what its exact name is), located to the immediate left of the Cross (x) at the top of the screen, and select Add to Home Screen

And hey presto! With these steps you should have created a mechanism that enables you to quickly & easily switch between your various playback devices, straight from the home screen, with a minimal number of steps involved.

Connectivity Testing TCP/IP

Go step-by-step accross the network. Ubuntu 20.

This is some real “bread and butter” for IT professionals.

The job of communication (i.e. payload data transmission) over a network-of-networks is a pretty complex problem. To solve this problem, network communications are broken out/down into protocol stacks; each protocol in the stack incrementally solving a part of the larger problem. The TCP/IP stack has emerged as the winner of the protocol wars, beating out the likes of SPX/IPX, AppleTalk, NetBIOS, etc. The situation is very much like VHS winning over Betamax in the videotape format war, and Blu-ray beating HD DVD in the optical disc war.

Each layer of the TCP/IP protocol stack solves a discrete part, or sub-dividision, of the communication problem, and represents an abstraction layer. Each distinct layer has its own job in getting data payload, from a running source process, one more step closer to the receiving destination process. As a helpful reminder, here are most of the the key ingredients for the internet protocol stack:

ProtocolJob
Application protocol (http(s), smtp, ssh, etc.)The job of the application protocol is to facilitate the actual payload data from running process on source host, to running process on destination. And thus, finally solving the communication problem.
Transmissiont Control Protocol (TCP) / User Datagram Protocol (UDP)The job of TCP/UDP is to get you to the correct running process “post box” or “pigeon hole” inside of the destination host.
Internet Protocol (IP) / Internet Control Messaging Protocol (ICMP)The job of IP is to get you from source host, on source network, over to destination host, on destination network.
Ethernet / Wi-FiThe job of Ethernet (physical + data-link) is to get you from one side of a single network / broadcast domain to the other side (usually a gateway).

TCP/IP protocols are open, collaborative, and progressive in nature. Individual standards documentation for each protocol are maintained by the Internet Engineering Task Force (IEFT) in the form of RFCs.

Local IP Config / Protocol Stack

Check you have IP address, subnet mask, gateway, and DNS servers.

# Your filename may be different
sudo vi /etc/neplan/00-installer-config.yaml

----

# This is the network config written by 'subiquity' 

network: 
  ethernets: 
    eth0: 
      addresses: 
      - 10.1.2.3/24 
      gateway4: 10.1.2.1 
      nameservers: 
        addresses: 
        - 10.2.2.41 
        - 10.2.3.41 
        search: 
        - example.com 
    eth1: 
      addresses: 
      - 10.2.3.12/24 
  version: 2 

----

sudo netplan apply

Check your network interface is “UP” and the IP config running against each interface looks correct.

ip a
hostname -I
ifconfig
ifup <ethx>
ifdown <ethx>

Check routing table information.

route -n
ip route
netstat -rn

Check your own public IP address.

curl ifconfig.co

Local Firewall

Ubuntu’s Uncomplicated Firewall command is a useful front for manipulating iptables.

sudo systemctl status ufw
sudo ufw status
sudo ufw status verbose
sudo ufw show raw
sudo ufw logging on [off]
cat /etc/ufw/*.rules

Ubuntu has a community wiki page for ufw.

DNS

Ultimately the destination host is reached by its IP address, but if DNS names are used then we should check they resolve to the correct IP address.

Check the DNS client’s configured name resolver servers.

nmcli device show <interfacename> | grep IP4.DNS
systemd-resolve --status | grep Current
cat /run/systemd/resolve/resolv.conf

Ensure your DNS name resolves to the correct IP address.

nslookup example.com

# To check against Google public DNS servers

nslookup example.com 8.8.8.8 [4.4.4.4]

ICMP / IP

Check the destination host IP interface is reachable, and also the route taken.

ping -c 3 1.2.3.4
ping -c www.example.com

tracert 1.2.3.4
tracert www.example.com

Transport

Check connection establishment to the receiving process.

# TCP
nc -z -v www.example.com 80
telnet www.example.com 22

# UDP
nc -u www.example.com 443

Inspect the state of network connections and related processes.

netstat -a
netstat --listening --numeric-ports --programs --tcp
netstat --numeric-ports --programs | grep 443
netstat -altp
netstat -tupan | grep -i http
netstat -ano | findstr "10389" | findstr "["

Securtiy Layer

Check TLS settings and certificate information.

openssl s_client -connect example.com:443 

Application Process

Check service status.

systemctl status <service_name>
systemctl is-enabled <service_name>
systemctl list-unit-files --state=enabled
systemctl list-unit-files --state=disabled
systemctl list-units --type=service [--state=running]
systemctl list-units --type=service --all

Check running process status.

sudo ps -ef | grep <string>

And there you have it! Hopefully the above steps will help you travel end-to-end on the communication journey to isolate / check many of the problems that you may encounter surrounding TCP/IP networking.

If you have any thoughts or suggestions please feel free to add into the comments below or contact Andrew using details on this site’s About page – we can always update this article to include anything that has been missed.

Favourite Tesla YouTubers

Keep up-to-date on Tesla. Subscribe, smash the like button, & ring the bell! 🙂

Solving the Money Problem – Steven Mark Ryan

Tesla Daily – Rob Maurer

Dave Lee on Investing – Dave Lee

HyperChange – Gali

Warren Redlich – Warren Redlich

Teslafaninsight – Greg

Oracle Investments – Oracle Tim

Dr. Know-it-all Knows it all – Dr Know-it-all

Farzad Mesbahi – Farzad Mesbahi

Whole Mars Catalog – Omar Qazi

Tesla Owners Silicoln Valley – John et al.

Tesla Fans – Tesla Motors Fans

Infowealth – EV News World

Jeff Roberts – Jeff Roberts

Tobias Lindh – Tobias Lindh

flybrandenburg – tessi-supply.com

My Tesla Weekend – Brian

Munro Live – Sandy Munro et al.

The Limiting Factor – Jordan Giesige

Good Soil Investment Management – Emmet Peppers et al.

Working with Cygwin

Bash on Windows.

Introduction

Cygwin is an open-source tool which offers a handy way to run Linux commands from a Windows desktop. 

Installation

Downoad the installer from https://cygwin.com/install.html and run it.  

Select the packages you want to install on top of the base installation. 

Browsing the Windows Host Filesystem

cd /cygdrive/c

Adding Common Packages

Run the installer again.

Search for the package you want. Here are some common ones:

PackageCommand(s)
ncnc
inetutilstelnet
curlcurl
rsyncrsync
jqjq

Working with Digital Certificates

On Ubuntu 20.

Digital certificates are a fundamental part of modern security. They assist in proving athenticity, meaning they help prove something, or someone, is who they claim to be.

File Formats (PFX & PKCS #12)

  

Certificates are stored and transferred as files.

Wikipedia explains the file types well: 

  

In cryptography, PKCS #12 defines an archive file format for storing many cryptography objects as a single file. It is commonly used to bundle a private key with its X.509 certificate or to bundle all the members of a chain of trust.” 

  

PKCS #12 is the successor to Microsoft‘s “PFX”;[5] however, the terms “PKCS #12 file” and “PFX file” are sometimes used interchangeably.” 

– https://en.wikipedia.org/wiki/PKCS_12 

  

View File Contents 

  

certutil -dump filename.pfx 

openssl.exe pkcs12 -info -in c:\temp\cert.pfx 

# For help: openssl pkcs12 -help 

  

Extract Certificate 

  

# -clcerts = only output client certificate 

openssl pkcs12 -in [yourfile.pfx] -clcerts -nokeys -out [drlive.crt] 

  

# -cacerts = only output CA certificates 

openssl pkcs12 in [yourfile.pfx] -cacerts -nokeys -out [drlive.bundle] 

  

Extract Private Key 

  

openssl pkcs12 -in [yourfile.pfx] -nocerts -out [drlive.key] 

openssl rsa -in [drlive.key] -out [drlive-decrypted.key] 

https://www.ibm.com/support/knowledgecenter/SSVP8U_9.7.0/com.ibm.drlive.doc/topics/r_extratsslcert.html

  

Create Intermediate Certificate Bundle 

  

From PFX file: 

# -cacerts = only output CA certificates 

openssl pkcs12 in [yourfile.pfx] -cacerts -nokeys -out [drlive.bundle] 

  

From CRT files: 

https://support.comodo.com/index.php?/Knowledgebase/Article/View/1145/1/how-do-i-make-my-own-bundle-file-from-crt-files

  

Check Certificate 

  

openssl x509 -in certname.crt -text -noout 

  

Check Key 

  

openssl rsa -in keyname.decrypted.key -check 

  

Check Key Matches Certificate 

  

MD5 output values should be matching: 

openssl x509 -noout -modulus -in certname.crt | openssl md5 

openssl rsa -noout -modulus -in keyname.decrypted.key | openssl md5 

Test a Live Site Certificate 

openssl s_client example.com:443 

https://www.sslshopper.com/ssl-checker.html