• Hello Universe!

    Hello Universe!

    Welcome to Dev-Doc-Ops!

    It’s a space for Engineering Technologists, focusing on how to work with modular, open-source, & service-oriented architectural building-blocks, to create modern platforms for digital services.

    Core interests & practices:

    Public cloud hosting (mainly AWS), Microservices/SOA/REST API, DevOps delivery, Agile/SCRUM ways of working, Cyber Security/DevSecOps practices, GitOps.

    There are likely to be posts on other interesting topics too; the common thread being technology & engineering.

  • Innovation Through Prototyping

    Innovation Through Prototyping

    Thoughts on bringing something new to life.

    I like the idea of seeing prototyping as a generic tool for bringing ideas into reality, and turning them into product.

    Regarding software, I consider a product to be a single instance of an application environment (you will need to define exactly where the application environment boundary lies for your particular software product).

    For it is the application environment, after all, that goes on to service the consumer.

    Prototyping is characterised by small, rapid, and iterative cycles of Plan-Do-Check-Act (PDCA). It personifies the “start small, think big, fail fast” start-up mentality, which is key to the success of most high-value, disruptive innovation & creativity.

    Here are some common developmental stages / phases I have observed innovative companies move their products through, in order to reach product maturity:

    Prototype –> PoC –> MVP –> Alpha release –> Beta release –> Wide release

    Stage / PhaseCharacterised By
    PrototypeA single, rapid loop iteration; used throughout.
    Proof-of-Concept (PoC)Demonstrates a hypothisis is true / feasible in reality.
    Minimum Viable Product (MVP)The smallest unit of value delivery, shippable to the product consumer.
    Alpha releaseInternal, small scale production.
    Beta releaseExternal, small scale production.
    Wide/full releaseLarge scale production; towards meeting total demand.

    Componets of the term ‘scale’ as it is used here:

    Scale = # product instances (e.g. unique application evironments) x speed/frequence of deployment

    When it comes to application environments, in contrast to hardware products, a deployment or delivery can be a brand new instance OR it could just be the updating of an existing instance (e.g. replacing some sub parts or components built from newer code).

    Software’s ‘bits’ are, it seems, easier to re-arrange than hardware’s ‘atoms’.

    Product maturity can also be characterised as the movement from scarcity of a product, to having that product in abundance. In this way the product is able to balance supply against the total demand that exists (assumes people want or need the product).

    In addtion, I find the concept of using prototyping-as-a-tool, when it comes to introducing new product functionality, does align well with both the Agile (i.e. easy to change/adapt) and DevOps (i.e. you build it, you run it) movements.

    ,

  • All Work Has Product

    All Work Has Product

    If not, why bother?

    Work is something which a human or machine (technology) does to affect some kind of change on our reality. That change may be something obvious (a new table has been made) or very subtle (a decision has been made). In all cases, something is made present which was not present before the work took place.

    If an official product name/title does not exist for what you are working on, then it should be possible to describe what the product of your work is, in your own words.

    If work has no product – think output – then the question should be raised as to whether it is valid and meaningful work to be doing.

    ,

  • Jenkins Should Be A Robot

    Jenkins Should Be A Robot

    Jenkins: The Build Machine.

    The Jenkins icon depicts an old-school Butler, with the quaint – and slightly amusing – suggestion being, we are employing another human being to go away and do some work for us.

    But to me, however, the real Jenkins is – whole or at least in part – actually the Machine-that-makes-the-machine!

    When combined with IaC, and automated API-based provisioning engines found in clouds such as AWS, we have all the digital instructions & machinery we need to create the end-machine (application environment) that will serve the end-consumer.

    Remeber, Jenkins makes the digital software products that go on to serve end-consumers; it isn’t an end-product iteslf.

    To re-interpret an old phrase: Jenkins is the means to the end, not the end itself.

    The analogy I would make is this: Jenkins is like a robot-arm (or die-cast machine) in a car factory. It is part of the machinery which builds the car, but it is not part of the car itself.

    Car factories are industrialised; typically they are designed to mass-manufacture at scale. They can stamp out millions of cars.

    My argument is, essentially, that ‘robot-arm’ is to car what ‘Jenkins’ is to application environment.

    Coming back to the original point of this article, Jenkins is a machine that can mass-manufacture software artifacts, not a human that hand-makes each one at low scale.

    Thus, at least by this logic, Jenkins should be a robot, not a human. 🙂

    ,

  • An Essay on Capital Economics

    An Essay on Capital Economics

    Let’s take a detour from technology for a short while…

    I have often mused over how the money supply grows.

    It stands to reason that when the amount of “stuff” (i.e. production output) changes then the money supply must also change in lock-step, in order to avoid any destabilisation from significant & unwanted amounts of inflation or deflation. Total money supply is in essence an abstract (technological) representation of total “stuff” supply (i.e. products, goods, and services) that make up the surplus by which Economics exists, to lay out a just mechanism for distribution to the populace at large (without surplus there is no Economics).

    Concider the main actors in the economic system: 1) Soverign, 2) Government, 3) Central Banks, 4) Retail Banks, 5) Businesses (surplus providers) & individuals (surplus consumers).

    The authorities (Soverign, Government, Central Bank) say to their Retail Banks:

    You can have the power to create brand new money out of thin air in the form of loans, however you must be able to attract that money back – with interest – to show that the recipient of the loan created new value for the economy (i.e. the societal surplus). This way, money supply should grow in-line with production output.

    And remember, each currency unit added to the money supply will reside in bank accounts, and so, due to fractional reserve banking, will be multiplied many times over (to account for screwges in the system that hoard wealth). Thus, demand is amplified and circling back to the original loan, this was to increase supply; they both were able to grow together.

    The ultimate purpose of this capitalistic economic mechanism, or system, is to move society away from scarcity and towards abundance.

    But what of the profit motive? This is simply a means to utilize the greed of the individual in a way that helps rather than harms the society around them. The deal is: individuals can have more of the surplus of society but in return they must also be adding back into the surplus in a net positive way, by putting in more extra value than they are taking out by consuming stuff (products & services). Therein lies the genius of capitalisim; aligning personal greed with societal improvement.

    If, for a period of time, supply has not kept up with demand (e.g. a supply chain shock), then here enters a second way to increase money supply. The Central Bank is empowered by the Government/Sovereign to offer increased interest rates to banks who store their on-hand deposits overnight with them. If Central Bank base rates are high enough then banks will elect to keep their money with the Central Bank rather than using it to tether a higher dept to equity ratio (remember banks can only be leveraged on their deposits by up to a certain ratio, usually by law). The net effect is banks lend less to the public, but also earn interest which is actually a net addition to the money supply (put into the wiser resource-allocating hands of the bankers, of course 😉 ).

    Finally, we have Quantitative Easing (QE) & Quantitative Tightening (QT). This is another way to increase money supply but with the notable difference that an account, or balance, is kept of exactly how much money – usually in the form of government or company bond purchases – has been added into the system. QE has the effect of stimulating demand by increasing liquidity throughout the system. This includes demand for assets such as property and securities (stocks), making asset prices go upwards, and asset owners feel richer and therefore more likely to spend. QT has the opposite effect; liquidity is pulled out of the system thereby decreasing demand.

    In conclusion, balance has to be maintained between supply and demand forces whilst, over time, the economy is grown. This is how we achieve prosperity through abundance for all. And – rather excitingly for technologists – the author believes technology plays an every increasing role in this beautiful process, as society becomes more and more advanced.

    ,

  • Perspectives on DevOps #1

    Perspectives on DevOps #1

    Short, succinct thoughts on the DevOps movement.

    DevOps is about accelerating customer value delivery.

    Some principles that can help enable this software mission:

    Balance planning & doing – both are needed, but in the right amount

    Ship often – small incremental changes

    Ship reliably – have a stable build machine/pipeline

    Fail fast, rollback safely – changes can be quickly & easily reversed

    Always listenting – responding continuously to feedback

    Always improving – have a product roadmap

    Also, a short comment on people teams/squads based upon statements I listened to recently but were recorded a long time ago from Elon Musk:

    The future is probabilistic.

    A team of people is a bunch of probabilities pointing in some direction.

    Aligning those probabilities to the same direction will maximise the chance of a successful outcome & future.

    ,

  • AWS REST API Gateway #1

    AWS REST API Gateway #1

    My journey to a basic achievement: “Hello World”

    As a learning stepping-stone for one of my recent projects, I decided to set myself the challenge of getting a simple AWS REST API Gateway up and running. In short, this service was to be implemented using Infrastructure-as-Code (IaC) to create a functional REST API Gateway in AWS that will return a message of “Hello World” on the API’s root public URL.

    The simplest REST API integration is called ‘MockIntegration‘. It does not require a separate backend, such as Lambda function or http endpoint, and so I chose to use this whilst getting started.

    The chosen IaC language/library was AWS Cloud Development Kit (CDK) for TypeScript.

    Along The Way

    I learned that it can sometimes be fairly straight-forward to create the items/objects you want within the graphical AWS web console, but it can be more challenging to then replicate what you have built using IaC.

    The main example of this was figuring out where to put the actual “Hello World” message in CDK code. I knew where to do this in the AWS console (under the GET method’s integration response) but was struggling to translate this knowledge into code.

    In the end it was the aws command line tool that came to my rescue, giving me the responseTemplate syntax that I needed. The following command revealed how to do it (after I had added the integration response in the AWS console first, and re-deployed the API stage so the config was live):

    > aws apigateway get-integration-response --rest-api-id <api_id> --resource-id <resource_id> --http-method GET --status-code 200

    The Code

    Anyhow, the code below assumes you have already run cdk init, to get the basic app/stack structure, and are inserting code into a stack constructor method.

    First, I created the main API instance:

    readonly api: RestApi

this.api = new RestApi(this, 'MyRestApi', {
   description: 'My REST API Gateway',
   endpointTypes: [aws_apigateway.EndpointType.REGIONAL],
   deployOptions: {
      stageName: 'development',
   }
})

    Next, the root MockIntegration is needed:

    this.api.root.addMethod('GET', new MockIntegration({
   integrationResponses: [
      {
         statusCode: '200',
         responseTemplates: {
            "application/json": "#set($inputRoot = $input.path('$'))\n{ message : \"Hello World\" }"
         }
      },
   ],
   passthroughBehaviour: PassthroughBehaviour.NEVER,
   requestTemplates: {
      'application/json': '{ "statusCode": 200 }',
   },
}), {
   methodResponses: [
      {
         statusCode: '200',
         responseModels: {
            'application/json': Model.EMPTY_MODEL
         }
      },
   ],
})

    The Result

    Finally, with the above code blocks in my stack constructor method, I had gathered together a sufficient amount of instruction for AWS to deploy my stack for me:

    > npx cdk deploy <stack_name> [--profile <profile_name>]

    Once successfully deployed, browsing to the output URL provided the desired “Hello World” message which I had been seeking.

    Goal achieved! Overall, as a learner of CDK for TypeScript, I found this to be a useful exercise to understand AWS API Gateway better.

    Useful Links

    Here are some links that helped me on my journey:

    Official AWS CDK Docs

    https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_apigateway-readme.html

    https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_apigateway.MockIntegration.html

    Other

    https://www.alexdebrie.com/posts/api-gateway-elements

    , , , ,

  • iPhone Playback Device Selector

    iPhone Playback Device Selector

    Using Apple Shortcuts.

    I am a recent convert to Apple. It started with a personal Macbook Air, but now I am well and truely into the Apple eco-system having a bunch of their hardware.

    When I swapped from Samsung/Android over to iPhone, one thing that seemed missing by default (I am willing to be corrected if wrong here) but is really useful is the ability to easily switch between playback devices, such as the iPhone itself, bluetooth headphones, and bluetooth speakers.

    A cool way I have found to achieve this from my home app screen is to use the Shortcuts app/feature that you get with iOS to create your own macro-like runbooks.

    Here is how I did it…

    Create A Shortcut for each Device

    1. Opent the Shortcuts app
    2. Click the Plus (+) button at the top right to add a new shortcut
    3. In the Search for apps and actions box, type “Playback”, then select the Scripting action for Change Playback Destination
    4. Choose a playback device from the list
    5. Give your new Shortcut a name at the top of the screen
    6. Press the Cross (x) button at the top right of the screen to save the new shortcut
    7. This should take you back to the All Shortcuts screen, where you should see your newly created shortcut
    8. Repeat for each one of your playback devices

    Create A Menu for the device shortcuts, Add to home screen

    Next we create a menu linked to an app icon which we can place on our home screen.

    1. As we did before, in the Shortcuts app, click the Plus (+) button at the top right to add a new shortcut
    2. This time, search for Choose from menu in the search box and add it
    3. Give your new menu a promt, something like “Select Device”
    4. Add a menu item for each of your playback devices and give them an intuative name (so you know which option is for which device)
    5. Then, for each playback device, search for and add a Run shortcut action, place the action under the respective menu item, and link the action to the relevant device shortcut (created earlier)
    6. Now you should have a viable menu structure, with options that each set a different playback device
    7. The final step is to create an icon to run your new menu from the home screen. Do this by pressing the options/preference/sliders button (not sure what its exact name is), located to the immediate left of the Cross (x) at the top of the screen, and select Add to Home Screen

    And hey presto! With these steps you should have created a mechanism that enables you to quickly & easily switch between your various playback devices, straight from the home screen, with a minimal number of steps involved.

    ,

  • Connectivity Testing TCP/IP

    Connectivity Testing TCP/IP

    Go step-by-step accross the network. Ubuntu 20.

    This is some real “bread and butter” for IT professionals.

    The job of communication (i.e. payload data transmission) over a network-of-networks is a pretty complex problem. To solve this problem, network communications are broken out/down into protocol stacks; each protocol in the stack incrementally solving a part of the larger problem. The TCP/IP stack has emerged as the winner of the protocol wars, beating out the likes of SPX/IPX, AppleTalk, NetBIOS, etc. The situation is very much like VHS winning over Betamax in the videotape format war, and Blu-ray beating HD DVD in the optical disc war.

    Each layer of the TCP/IP protocol stack solves a discrete part, or sub-dividision, of the communication problem, and represents an abstraction layer. Each distinct layer has its own job in getting data payload, from a running source process, one more step closer to the receiving destination process. As a helpful reminder, here are most of the the key ingredients for the internet protocol stack:

    ProtocolJob
    Application protocol (http(s), smtp, ssh, etc.)The job of the application protocol is to facilitate the actual payload data from running process on source host, to running process on destination. And thus, finally solving the communication problem.
    Transmissiont Control Protocol (TCP) / User Datagram Protocol (UDP)The job of TCP/UDP is to get you to the correct running process “post box” or “pigeon hole” inside of the destination host.
    Internet Protocol (IP) / Internet Control Messaging Protocol (ICMP)The job of IP is to get you from source host, on source network, over to destination host, on destination network.
    Ethernet / Wi-FiThe job of Ethernet (physical + data-link) is to get you from one side of a single network / broadcast domain to the other side (usually a gateway).

    TCP/IP protocols are open, collaborative, and progressive in nature. Individual standards documentation for each protocol are maintained by the Internet Engineering Task Force (IEFT) in the form of RFCs.

    Local IP Config / Protocol Stack

    Check you have IP address, subnet mask, gateway, and DNS servers.

    # Your filename may be different
sudo vi /etc/neplan/00-installer-config.yaml

----

# This is the network config written by 'subiquity' 

network: 
  ethernets: 
    eth0: 
      addresses: 
      - 10.1.2.3/24 
      gateway4: 10.1.2.1 
      nameservers: 
        addresses: 
        - 10.2.2.41 
        - 10.2.3.41 
        search: 
        - example.com 
    eth1: 
      addresses: 
      - 10.2.3.12/24 
  version: 2 

----

sudo netplan apply

    Check your network interface is “UP” and the IP config running against each interface looks correct.

    ip a
hostname -I
ifconfig
ifup <ethx>
ifdown <ethx>

    Check routing table information.

    route -n
ip route
netstat -rn

    Check your own public IP address.

    curl ifconfig.co

    Local Firewall

    Ubuntu’s Uncomplicated Firewall command is a useful front for manipulating iptables.

    sudo systemctl status ufw
sudo ufw status
sudo ufw status verbose
sudo ufw show raw
sudo ufw logging on [off]
cat /etc/ufw/*.rules

    Ubuntu has a community wiki page for ufw.

    DNS

    Ultimately the destination host is reached by its IP address, but if DNS names are used then we should check they resolve to the correct IP address.

    Check the DNS client’s configured name resolver servers.

    nmcli device show <interfacename> | grep IP4.DNS
systemd-resolve --status | grep Current
cat /run/systemd/resolve/resolv.conf

    Ensure your DNS name resolves to the correct IP address.

    nslookup example.com

# To check against Google public DNS servers

nslookup example.com 8.8.8.8 [4.4.4.4]

    ICMP / IP

    Check the destination host IP interface is reachable, and also the route taken.

    ping -c 3 1.2.3.4
ping -c www.example.com

tracert 1.2.3.4
tracert www.example.com

    Transport

    Check connection establishment to the receiving process.

    # TCP
nc -z -v www.example.com 80
telnet www.example.com 22

# UDP
nc -u www.example.com 443

    Inspect the state of network connections and related processes.

    netstat -a
netstat --listening --numeric-ports --programs --tcp
netstat --numeric-ports --programs | grep 443
netstat -altp
netstat -tupan | grep -i http
netstat -ano | findstr "10389" | findstr "["

    Securtiy Layer

    Check TLS settings and certificate information.

    openssl s_client -connect example.com:443

    Application Process

    Check service status.

    systemctl status <service_name>
systemctl is-enabled <service_name>
systemctl list-unit-files --state=enabled
systemctl list-unit-files --state=disabled
systemctl list-units --type=service [--state=running]
systemctl list-units --type=service --all

    Check running process status.

    sudo ps -ef | grep <string>

    And there you have it! Hopefully the above steps will help you travel end-to-end on the communication journey to isolate / check many of the problems that you may encounter surrounding TCP/IP networking.

    If you have any thoughts or suggestions please feel free to add into the comments below or contact Andrew using details on this site’s About page – we can always update this article to include anything that has been missed.

    , ,

  • Favourite Tesla YouTubers

    Favourite Tesla YouTubers

    Keep up-to-date on Tesla. Subscribe, smash the like button, & ring the bell! 🙂

    Solving the Money Problem – Steven Mark Ryan

    Tesla Daily – Rob Maurer

    Dave Lee on Investing – Dave Lee

    HyperChange – Gali

    Warren Redlich – Warren Redlich

    Teslafaninsight – Greg

    Oracle Investments – Oracle Tim

    Dr. Know-it-all Knows it all – Dr Know-it-all

    Farzad Mesbahi – Farzad Mesbahi

    Whole Mars Catalog – Omar Qazi

    Tesla Owners Silicoln Valley – John et al.

    Tesla Fans – Tesla Motors Fans

    Infowealth – EV News World

    Jeff Roberts – Jeff Roberts

    Tobias Lindh – Tobias Lindh

    flybrandenburg – tessi-supply.com

    My Tesla Weekend – Brian

    Munro Live – Sandy Munro et al.

    The Limiting Factor – Jordan Giesige

    Good Soil Investment Management – Emmet Peppers et al.

    ,
    ,

  • Working with Cygwin

    Working with Cygwin

    Bash on Windows.

    Introduction

    Cygwin is an open-source tool which offers a handy way to run Linux commands from a Windows desktop. 

    Installation

    Downoad the installer from https://cygwin.com/install.html and run it.  

    Select the packages you want to install on top of the base installation. 

    Browsing the Windows Host Filesystem

    cd /cygdrive/c

    Adding Common Packages

    Run the installer again.

    Search for the package you want. Here are some common ones:

    PackageCommand(s)
    ncnc
    inetutilstelnet
    curlcurl
    rsyncrsync
    jqjq

    , ,