Category Archives: Between the Brackets

gitlab CI

Getting into continuous integration can be a little intimidating, here’s a good gitlab-ci.yml that I’ve adjusted over time and is pretty much drop in for my development cycle. You’ll have to add your own variables in a projects CI settings and do some SSH setup for rsync to work, I’ll cover that later.

stages:
  - test
  - build
  - user-test
  - deploy

test:
  stage: test
  before_script:
  # Find the SSH Agent
  - 'which ssh-agent'
  # Run The Agent
  - eval $(ssh-agent -s)
  # Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
  - ssh-add <(echo "$SSH_PRIVATE_KEY")

  script:
    - echo "Make Sure Commit SHA Exists, Use for revision hashing"
    - echo $CI_COMMIT_SHA
    - echo "Test Deploy To Preview"
    - rsync --update --dry-run --checksum -rlvzub --exclude ".*" --exclude "*.tgz" --exclude "Makefile.*" --backup-dir="backups/$CI_JOB_ID" -e "ssh -i ~/.ssh/id_rsa.pub" _src/ gitlab-runner@linux3:/var/www/mc-dev/$DEPLOYMENT_PREVIEW_DESTINATION/_preview
    - echo "Dry Run Deployment On Linux3"
    # - ssh gitlab-runner@linux3 -tt
    - rsync --update --dry-run --checksum -rlvzub --exclude ".*" --exclude "*.tgz*" --exclude "Makefile.*" --backup-dir="backups/$CI_JOB_ID" -e "ssh -i ~/.ssh/id_rsa.pub" _src/ gitlab-runner@linux3:/var/www/$DEPLOYMENT_DESTINATION

build:
  stage: build
  script: 
    - echo "Building The App"
    - echo $CI_COMMIT_SHA

user-test: 
  stage: user-test
  before_script:
    # Find SSH Agent
    - 'which ssh-agent'
    # Run the Agent
    - eval $(ssh-agent -s)
    #Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
    - ssh-add <(echo "$SSH_PRIVATE_KEY")
  script:
    # - echo "Update Local Preview"
    # - rsync --update --checksum -rlvzu --exclude ".*" --exclude "*.tgz" --exclude "Makefile.*" -e "ssh -i ~/.ssh/id_rsa.pub" _src/ _preview
    - echo "Create Revision Hashes"
    - sed -i "s/@@hash/$CI_COMMIT_SHA/g" _src/index.php
    - echo "Deploying to the preview folder"
    - rsync --update --checksum -rlvzub --exclude ".*" --exclude "*.tgz" --exclude "Makefile.*" --backup-dir="backups/$CI_JOB_ID" -e "ssh -i ~/.ssh/id_rsa.pub" _src/ gitlab-runner@linux3:/var/www/mc-dev/$DEPLOYMENT_PREVIEW_DESTINATION/_preview
    
  when: manual
  allow_failure: true

deploy_prod:
  stage: deploy
  before_script:
    # Find the SSH Agent
    - 'which ssh-agent'
    # Run The Agent
    - eval $(ssh-agent -s)
    # Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
    - ssh-add <(echo "$SSH_PRIVATE_KEY")

  script:
    - echo "Deploy App to Linux 3"
    # - ssh gitlab-runner@linux3 -tt
    - echo "Create Revision Hashes"
    - sed -i "s/@@hash/$CI_COMMIT_SHA/g" _src/index.php
    - echo "Copying Files"
    - rsync --update --checksum -rlvzub --exclude ".*" --exclude "*.tgz*" --exclude "Makefile.*" --backup-dir="backups/$CI_JOB_ID" -e "ssh -i ~/.ssh/id_rsa.pub" _src/ gitlab-runner@linux3:/var/www/$DEPLOYMENT_DESTINATION

  environment:
    name: production
    url: http://linux3/$DEPLOYMENT_DESTINATION
  only:
    - master
    - merge-requests
  # Uncomment below to turn on manual deployment
  # when: manual
  allow_failure: false

 

Keep an eye out for more coverage of setting up rsync over ssh.

Markdown -> Email

I love markdown, I like the simplicity of it, the consistent formatting and all the edge features that are possible. So sifting through my snippet collection I found this beaut in my GitHub and my Gitlab snippet collections so it must be good right?


#!/usr/bin/env python

'''
Send an multipart email with HTML and plain text alternatives. The message
should be constructed as a plain-text file of the following format:

From: Your Name 
To: Recipient One 
Subject: Your subject line
---
Markdown content here
The script accepts content from stdin and, by default, prints the raw
generated email content to stdout.
Preview your message on OS X by invoking the script with `-p` or
`--preview`, and it will open in your default mail client.
To send the message, invoke with `-s` or `--send`. You must have a
JSON file in your home directory named `.markdown-to-email.json`
with the following keys:
{
"username": "smtp-username",
"smtp": "smtp.gmail.com:587",
"password": "your-password"
}
Enjoy!
'''
import os
import sys
import json
import argparse
import smtplib
import subprocess

from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText

try:
import pygments
import markdown
except ImportError:
print 'This script requires pygements and markdown to be installed.'
print 'Please:'
print ' pip install pygments markdown'
sys.exit(0)

# define arguments
parser = argparse.ArgumentParser(description='Format and send markdown-based emails.',
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__)
parser.add_argument('-p', '--preview', action='store_true',
help='Preview the email in Apple Mail.')
parser.add_argument('-s', '--send', action='store_true',
help='Send the email using your configuration.')
args = parser.parse_args()

# read in raw message content
raw_content = sys.stdin.read()

# split out the headers from the markdown message body
header_content, markdown_content = raw_content.split('--', 1)

# render the markdown into HTML
css = subprocess.check_output(['pygmentize', '-S', 'default', '-f', 'html'])
markdown_content = markdown_content.strip()
html_content = markdown.markdown(markdown_content, ['extra', 'codehilite'])
html_content = ''+css+''+html_content

# create a multipart email message
message = MIMEMultipart('alternative')

# parse the headers
headers = {}
for line in header_content.strip().split('\n'):
if not line.strip(): continue
key, value = line.split(':', 1)
headers[key.strip()] = value.strip()

# set the headers
message['To'] = headers.get('To', '')
message['From'] = headers.get('From', '')
message['Subject'] = headers.get('Subject', 'No subject')

# attach the message parts
message.attach(MIMEText(markdown_content, 'plain'))
message.attach(MIMEText(html_content, 'html'))

if args.send:
to = message['To'].split(', ')

with open(os.path.expanduser('~/.markdown-to-email.json'), 'rb') as f:
config = json.loads(f.read())
server = smtplib.SMTP(config['smtp'])
server.starttls()
server.login(config['username'], config['password'])
server.sendmail(message['From'], to, message.as_string())
server.quit()
elif args.preview:
open('/tmp/preview.eml', 'w').write(message.as_string())
os.system('open -a Mail /tmp/preview.eml')
else:
print message.as_string()

 

It has some external dependencies but creating rich html emails from simple markdown base is pretty sweet in my book.

For when you need more info

One of the toughest things is when you try to help someone and there isn’t enough information to start formulating a response. I come across this more often than not on stackoverflow and I keep this snippet close;

For the help you want, you need to share your Minimal, Complete, and Verifiable example HTML, CSS, and JavaScript. This way we can reproduce your problem and, from there, offer a useful and practical solution. If needed create a fiddle at [jsfiddle.net](jsfiddle.net) and review [How To Ask](https://stackoverflow.com/help/how-to-ask)

SOF can be a little less than inviting sometimes and I find this helps everyone get the help they’re looking for.