AWS Template Creation by Script

During an AWS architecture class, we had to create and launch an AWS Stack. Within the stack, it was Infrastructure as Code, but the actual launch of the stack was done at the console. Once upon a time, I knew I had worked with stack creation as IaC. I dug back through some of my old examples and found the code (below) that I used to create the stack, along with some of the variables.

The Code

Line numbers are for reference. Note that this is a single bash shell block (hence the “\” at the end of each line starting in line 2.

1.  cfn_stack_name="${JOB_NAME}-${pipeline_instance_id}"
2.  cfn_stack_id=$(aws cloudformation create-stack \
3.     --disable-rollback \
4.     --region $region \
5.     --stack-name "$cfn_stack_name" \
6.     --template-body "file://${cfn_template_path}" \
7.     --parameters ParameterKey=amiID,ParameterValue=$baseami \
8.         ParameterKey=vpcID,ParameterValue=$vpc \
9.         ParameterKey=subnetID,ParameterValue=$subnet \
10.        ParameterKey=keypairName,ParameterValue=$jenkins_key_name \
11.    --tags Key=BuiltBy,Value="Jenkins_$(hostname)" \
12.    --tags Key=AWS_OP_ENV,Value="$aws_op_env" \
13.    --tags Key=Server,Value="$server_function" \
14.    --tags Key=System,Value="$system" \
15.    --query 'StackId' --output text)
16. max_waitime=600
17. wait_interval=5
18. # wait until the stack is created
19. echo "Waiting for CFN stack to be created..."
20. time monitor_stack --region "$region" --stack "$cfn_stack_name"
21. cfn_instance_id=$(aws cloudformation describe-stacks --region $region --stack-name="$cfn_stack_name" --query 'Stacks[0].Outputs[0].OutputValue' --output text)
22. echo "CGN stack created!"

The other thing to note is you need to have the AWS CLI installed in your build environment for this to work. In most cases, you will be building this inside AWS, so the CLI will be available to you.

The Explanation

In the code starting on line 1:


The JOB_NAME and pipeline_instance_id are generated by the Jenkins job. You can name it however you want, that was just what we used. We originally started with just date/time stamps.

Line 2 begins the actually stack creation:

cfn_stack_id=$(aws cloudformation create-stack

The cfn_stack_id is generated at the end of the code block: --query 'StackID' --output text. The syntax may be old, check the documentation for the correct call for the StackID. The rest of the data is necessary to define the stack.

Most of the variables are defined higher up in the script, most based on calls to a DymanoDB instance where we would store various bits of data that may or may not have changed throughout the build process, or as defined by the customer. We also saved the stack name in that same DB system so we could tear it down later.

Finally we wrapped it with a timer value. This may need to be adjusted based on the speed of the environment or number of variables you are pushing into the stack. You want the system to error out if things are too busy, otherwise the script will hang and the build server will appear to be stuck. We also had some additional verbiage at the bottom of the script that pushed text to the log file/console output so you could see it succeed as shown in lines 18 – 22.

One other thing to note is that the stack also launches an AMI (again pulled from reference). Once this stack and associated AMI are up, the next part of the pipeline starts. This could populate the AMI, test it, turn it into a Jenkins build server, whatever was necessary. The key here is it is all code.

Installing MediaWiki on Ubuntu 18

A buddy send a request. He was installing MediaWiki on Ubuntu and he was having issues so he asked me to take a look. I reviewed a link on Linux Support and HowtoForge on installing MediaWiki, and found them to be a tad dated. So, I went through the installation myself, and here is how I installed it.

All steps are done as an sudoer or as the root user. I did this on AWS with a Ubuntu 18.04 minimal base image. I assume you know how to log into a console. I used Apache. You can use Nginx, but the server directions are different and I did not have a chance to try them out.

Update the OS

sudo apt-key adv --recv-keys --keyserver hkp:// 0xF1656F24C74CD1D8
sudo add-apt-repository "deb [arch=amd64,arm64,ppc64el] $(lsb_release -cs) main"
sudo apt-get update
sudo apt-get upgrade

Install basic packages

sudo apt-get install -y apache2 software-properties-common
sudo apt -y install mariadb-server mariadb-client
sudo apt install php libapache2-mod-php
sudo apt-get install imagemagick php7.2-fpm php7.2-intl php7.2-xml php7.2-curl php7.2-gd php7.2-mbstring php7.2-mysql php7.2-mysql php-apcu php7.2-zip

Once PHP is installed you will get a notice similar to:

NOTICE: Not enabling PHP 7.2 FPM by default.
NOTICE: To enable PHP 7.2 FPM in Apache2 do:
NOTICE: a2enmod proxy_fcgi setenvif
NOTICE: a2enconf php7.2-fpm

I enabled it after the fact and it worked. You can do it now or later as you desire.

Modify PHP settings (Optional)

If you are putting your server into production, use the following settings initially. If you are just looking around, the default php.ini settings are fine except for the timezone settings. You should set the timezone appropriately.

For production, edit /etc/php/7.2/apache2/php.ini and make the following changes:

memory_limit = 256M
upload_max_filesize = 100M
max_execution_time = 360
date.timezone = America/New York

Run the secure installation for MariaDB (Optional)

If you are running a production server, you should do a secure installation.

sudo mysql_secure_installation

Create the MediaWiki table space

Login to MariaDB

mariadb -u root -p

And create the MediaWIki user and db as follows

CREATE USER 'media'@'localhost' IDENTIFIED BY 'password';
GRANT ALL ON mediadb.* TO 'media'@'localhost' IDENTIFIED BY 'password' WITH GRANT OPTION;

Where password is a secure password. This will be put into the MediaWiki configuration later, so do not forget it. The database mediadb and user media can be anything you want them to be.

Edit Apache’s site configuration

You will need to add MediaWiki to the site configuration. Create a new file called mediawiki.conf

sudo vi /etc/apache2/sites-available/mediawiki.conf

And add the following:

<VirtualHost *:80>
DocumentRoot /var/www/html/mediawiki/
<Directory /var/www/html/mediawiki/>
Options +FollowSymLinks
AllowOverride All
ErrorLog /var/log/apache2/media-error_log
CustomLog /var/log/apache2/media-access_log common

Where the ServerAdmin variable should be real email address and the ServerName should be the domain name of the server. Also, ensure that the DocumentRoot is correct. If you only want to use MediaWiki, you can set the DocumentRoot to /var/www/html, but you have to modify a step below as well.

Restart everything

Do not restart the server yet! Instead, restart the key services.

sudo a2ensite mediawiki.conf
sudo a2enmod rewrite
sudo systemctl start apache2
sudo systemctl enable apache2
sudo systemctl start mariadb
sudo systemctl enable mariadb

Download the current MediaWiki source

From the MediaWiki site, make sure you have the correct version. As of this writing, it is: mediawiki-1.33.1

Change to a temporary directory, download, untar, and move the file to the web server:

tar zxvf mediawiki-1.33.1.tar.gz 
sudo mkdir -p /var/www/html/mediawiki
sudo mv mediawiki*/* /var/www/html/mediawiki

If you modified the DocumentRoot in the Apache configuration to /var/www/html, you will need to modify the command above. You will only need to move the contents of the base mediawiki folder:

sudo mv mediawiki*/* /var/www/html

Point your browser at the web site

Depending on your confirmation you can either use localhost or the hostname of your server. If you use the mediawiki folder option, you have to put the folder on the end.


Good luck!

Web Links

MAX and Human Errors

What Really Brought Down the Boeing 737 Max? – The New York Times

In the drama of the 737 Max, it was the decisions made by four of those pilots, more than the failure of a single obscure component, that led to 346 deaths and the worldwide grounding of the entire fleet.

I am not a pilot, and I have never been at the controls of an airplane. This very long article does go into a number of issues surrounding a complicated piece of technology. Take a read. It does not take any responsibility off of Boeing, but it certainly does not make them out to be the only villain in the story.

Checks and Balances

Trump impeachment: Lindsey Graham will ‘not pretend to be a fair juror’

Asked if it was appropriate for him as a prospective juror to be discussing the case in such terms, he said: “Well, I must think so because I’m doing it.”

Once upon a time, the Founding Fathers instituted a provision to remove a sitting President for High Crimes and Misdemeanors. Since that time four US Presidents have come under those provisions. Prior to this case, the Senators who should be trying the case (as jurors) kept their opinions to themselves. Not this time. This time, they are coming out and telling us exactly how they are going to vote. Before even hearing one witness. Before even seeing one document. They feel that this is a hit job and this is how they are going to vote.

If they were a real jury of their peers, they would be dismissed, at the very lest. I am sure there would be others ramifications. But these are United States Senators. The Founding Fathers are wondering what has happened to the Republic they strove so hard to create.

That is a big hole

Scientists have discovered deepest point on land | WTOP

The trough is about 3.5 km (about 2 miles) below sea level but there is no ocean water there. Instead, it is filled with ice flowing from the interior of the ice sheet towards the coast. The trough measures about 100 km in length and is 20 km wide, according to the study.

Think about how long 100 km is. The District of Columbia is 16 km on a side. According to Wikipedia, 100 km is 9/10 as long as the English Channel and not quite as wide as the narrowest point. And it is on land! Think about that with your morning coffee.

Review of the YSmart TIPEN

YSmart introduced a new pen on Kickstarter easier in 2019 (and it is now available to purchase on Indiegogo. Since I have not met a pen that I did not like, especially one made out of virtually indestructible, go anywhere metal, I bought into the program. My pens arrived this week and here is my review.

First, this pen is tiny. I mean really, really small. For comparison, in the image above, we have the YSmart TIPEN resting against the ruler, end to end it is barely 2 inches long. For scale, above we have the Fisher Bullet Space Pen, a basic black marker, and a standard, freshly sharpened number 2 pencil. However, uncapped, the pen is even smaller.

Unlike the Bullet pen, which will take its cap on the back, and gives you an extra inch or so, the TIPEN cap will not fit on the back, leaving the pen at 2 inches. For those of us with long fingers, this becomes a bit of a problem when writing, especially if you are used to resting the pen against your finger.

This also impacts the quality of your writing, especially over time. You will not be writing long epistles with the TIPEN, but it is useful to have around for quick notes and shopping lists.

The ink is similar in feel to the Fisher refill, which is why I chose it for comparison. It is not a ballpoint ink per se, nor is it a gel ink (my preferred ink in non-fountain pens). It writes smoothly and with no skip once started. YSmart claims additionally that the nib is unbreakable and suitable for opening packages, paint cans, and non-writing functions.

For an EDC pen, it would not be my first choice. Despite its slightly larger size, I would select the Bullet pen, or its brother, the Trekker pen with a key chain ring attachment, but for an emergency pen, the TIPEN is a good choice. You can put it on your key ring and forget about it until you need it.


Lawmakers unveil details of ‘historic’ federal paid parental leave benefits | Federal News Network

The annual defense policy bill, if passed by both chambers of Congress and signed into law by the president, would grant federal employees up to 12 weeks of paid leave for the birth, adoption or foster of a new child.

If you live in the United States, and you are an employee of the United States Federal Government, and you are planning to have a family, this is a wonderful benefit, assuming the bill is actually passed, which with this current government, is doubtful. However, to call it historic, or even wonderful, falls well short of the mark.

Other backslapping terms cited in the article include watershed, life-changing, and monumental. If the bill passes, this will go into effect in October of 2020, more than a year from now. Please make a note of that in your family planning.

Why am I so contemptuous of this policy? For starters, this only applies to employees of the Federal Government. Without a calculator, I cannot accurately estimate the impact. Still, the number of people that this will benefit is a fraction of a percent of the overall workforce in the United States when you consider those of childbearing or family starting years and those who are actual Federal Employees. This does not cover contractors or anyone else that toils for a paycheque in the United States.

I am also derisive of this policy because it still falls considerably short of the policies for other First World/Developed nations around the world. According to the United Nations, of 193 countries, only a handful do not have any national paid parental leave law. Guess who they are? New Guinea, a few South Pacific island nations and the United States. The Federal law would align the benefit for Federal Employees with the basic minimums that are already prevalent in the world, which means we are not on par with countries like Sudan (oh, wait, never mind, everyone in Sudan gets 13 weeks).

Most of the developed nations start at 26 weeks and go up from there, with a guarantee that your job will still be there should you decide to return. Let me say that again. If you go on maternity leave, most policies guarantee that the employee will have a job when they return from their leave. It is not clear that such a guarantee is in this bill. I will need to check that, but I would hope it is. Of course, this bill was supposed to be a larger bill that also provided for the care of sick family members, a growing problem in the United States as the cost of health care skyrockets, and the population is aging at a rapid rate. That is still a significant hurdle that many people, not just Federal Employees, need to overcome on a daily basis.

I congratulate the Federal Government. An organization that has been the whipping boy of both the President and Congress, where hiring the best and the brightest has never been easy, now has a benefit worth writing home about. If the bill gets passed. And signed. And not watered down in committee, and any of a dozen other things that could happen before October of next year. Now, what about the rest of the population?

I Am Not Filling Out Your Survey

And if I do, you won’t like the outcome.

Grade inflation, or rather star inflation, is rampant in online shopping. I blame Amazon. But it has gotten so carried away that everyone from the checkout clerk at the grocery store to the guy that sends me simple screws is asking for my review of their performance, or their product. Let’s face it; I do not have the time.

If you keep insisting, I am going to grade you precisely the same way I graded my employees. I used to work for a company with a hideous annual review process. If you did your job, you got a three out of five. If you aspired for a higher grade, there were strict criteria. To get a four, you had to be recognized as an expert or leader by other people in the company. We had over seven hundred people. To be recognized in this way, it rarely happened. We used to say god had you on speed-dial. To get a five, you had to be recognized as a leader in your industry. God had you on speed-dial.

I am going to take the same tact with surveys. If it does the job, you get a three. To get more stars beyond that, you are going to have to rock my world. Second, if you sell me a product, do not ask me for my initial opinion. I probably have not had time to use it effectively. Ask me again in another month. Or another six months. That will allow me to evaluate your product correctly. Frankly, unless it changes my world, you will still only get three stars. The number of things that have risen to that level through my life is so minimal that it cannot be counted.

Just stop asking.

When Privacy and Reality Interconnect

His privacy being paramount, Kelly grudgingly chooses to head into Columbia every so often, rather than cede his data to Google or turn over his purchase history to another online retailer. “I’m just not sure why Google needs to know what breakfast cereal I eat,” the 51-year-old said. Washington Post

There are a couple of things to notice here.

First: Google is not the only company out there snarfing up your data. Zuckerbergland apps, Verizon (you know, AOL, Yahoo, Tumblr), Microsoft (Linkedin, Bing, all those Microsoft apps like Word, etc) are only some of them.

Second: Most websites have some form of tracking software on them, and they can be related to any of the three or more listed above.

Third: Despite what the EU would have you believe, GDPR is not your salvation, as many websites, in the small print, outside the EU say this site not intended for consumption by people in the EU which means that the GDPR has zero impact.

And realistically, if you do not want to be tracked, there is only one way to avoid it. Stay off the Internet. And that includes no smart devices (there is tracking software on them too), no credit cards (who do you think came up with the idea of tracking purchases), and no cheques. In fact, depending where you live, you are being watched by CCTV cameras, where the video is uploaded and searched for malcontents, using AI and facial recognition software. If you travel, you are tracked whether by planes, trains, or automobile (toll plazas, rest stops…). Let’s face it, unless you are a hermit, you have no privacy.

And ironically, we all know that Mr Kelly, who is 51 years-old, likes to eat Bob’s Red Mill muesli cereal. So his privacy is now shot too, because he talked to a reporter, and the story ended up…on the Internet.