Setting up Discourse with Fastly as a CDN provider and SSL

The following is a copy of what I published in a question on meta.discourse.org about “Enable a CDN for your Discourse while working on discuss.webplatform.org.

Setup detail

Our setup uses Fastly, and leverage their SSL feature. Note that in order for you to use SSL too, you’ll have to contact them to have it onto your account.

SEE ALSO this post about Make Discourse “long polling” work behind Fastly. This step is required and is a logical next step to this procedure.

In summary;

  • SSL between users and Fastly
  • SSL between Fastly and “frontend” servers. (That’s the IP we put into Fastly hosts configuration, and are also refered to as “origins” or “backends” in CDN-speak)
  • Docker Discourse instance (“upstream“) which listens only on private network and port (e.g. 10.10.10.3:8000)
  • More than two publicly exposed web servers (“frontend“), with SSL, that we use as “backends” in Fastly
  • frontend server running NGINX with an upstream block proxying internal upstream web servers that the Discourse Docker provides.
  • We use NGINX’s keepalive HTTP header in the frontend to make sure we minimize connections

Using this method, if we need to scale, we only need add more internal Discourse Docker instances, we can add more NGINX upstream entries.

Note that I recommend to use direct private IP addresses instead of internal names. It removes complexity and the need to rewrite Hosts: HTTP headers.

Steps

Everything is the same as basic Fastly configuration, refer to setup your domain.

Here are the differences;

  1. Setup your domain name with the CNAME Fastly will provide you (you will have to contact them for your account though), ours is like that ;

    discuss.webplatform.org.  IN  CNAME  webplatform.map.fastly.net.
    
  2. In Fastly pannel at Configure -> Hosts, we tell which publicly available frontends IPs

    Notice we use port 443, so SSL is between Fastly and our frontends. Also, you can setup Shielding (which is how you activate the CDN behavior within Fastly) by enabling it on only one. I typically set it on the one I call “first”.

    Fastly service configuration, at Hosts tab

  3. In Fastly pannel Configure -> Settings -> Request Settings; we make sure we forward X-Forwarded-For header. You DONT need this; you can remove it.

    Fastly service configuration, at Settings tab

  4. Frontend NGINX server has a block similar to this.

    In our case, we use Salt Stack as the configuration management system, it basically generates the Virtual Hosts for us as using Salt reactor system. Every time a Docker instance would become available, the configuration will be rewritten using this template.

    • {{ upstream_port }} would be at 8000 in this example

    • {{ upstreams }} would be an array of current internal Docker instances, e.g. ['10.10.10.3','10.10.10.4']

    • {{ tld }} would be webplatform.org in production, but can be anything else we need in other deployment, it gives great flexibility.
    • Notice the use of discoursepolling alongside the discourse subdomain name. Refer to this post about Make Discourse “long polling” work behind Fastly to understand its purpose

      upstream upstream_discourse {
      {%- for b in upstreams %}
          server    {{ b }}:{{ upstream_port }};
      {%- endfor %}
          keepalive 16;
      }
      
      server {
          listen      443 ssl;
          server_name discoursepolling.{{ tld }} discourse.{{ tld }};
      
          root    /var/www/html;
          include common_params;
          include ssl_params;
      
          ssl                 on;
          ssl_certificate     /etc/ssl/2015/discuss.pem;
          ssl_certificate_key /etc/ssl/2015/201503.key;
      
          # Use internal Docker runner instance exposed port
          location / {
              proxy_pass             http://upstream_discourse;
              include                proxy_params;
              proxy_intercept_errors on;
      
              # Backend keepalive
              # ref: http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive
              proxy_http_version 1.1;
              proxy_set_header Connection "";
          }
      }
      

    Note that I removed the include proxy_params; line. If you have lines similar to proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;, you don’t need them (!)

Creating a new Ubuntu Salt master from the terminal using Cloud-Init

If you run on Virtual Machines on a provider that runs OpenStack you can also leverage a component that’s made to automatically install softwares at creation time. With this, you can any new node in your cluster, including the salt-master in a few terminal commands.

Before starting out, you need to make sure your cloud provider runs Open Stack and has Cloud-Init enabled. To check it out, look in the “Launch instance” dialog to create a new VM a tab titled “Post-Creation”, it might just simply work.

img

Cloud-Init is basically reading a manifest that declares what’s the specifics of the new VM and is part of the conversion from the initial image OpenStack into the specific instance you will be using. You can follow those two articles that describes well how Cloud-Init works.

[Cloud-Init] is made in a way that it handles distribution specific package installation details automatically.

The following is specific to an Ubuntu server VM, but you might need to adjust the package names to match your current server distribution as those tools are getting more and more popular in the industry.

Before testing out on a new VM, you could also check from an existing instance and ask through an HTTP request what was the current instance’ post-creation script using cURL.

Note that the IP address you see below is a virtual interface provided by OpenStack but can be navigated through HTTP, try it out like this;

curl http://169.254.169.254/openstack/
2012-08-10
2013-04-04
2013-10-17

If you see a similar output, you can ask what was the post-creation configuration (“userdata”) it used at creation time. You can dig the tree, here’s how you can find it in an OpenStack (CURRENT VERSION NICKNAME) cluster.

For instance, my a salt master would have the following configuration;

curl http://169.254.169.254/openstack/2013-10-17/user_data

#cloud-config
manage_etc_hosts: false
manage-resolv-conf: false
locale: en_US.UTF-8
timezone: America/New_York
package_upgrade: true
package_update: true
package_reboot_if_required: true

ssh_import_id: [saltstack]
apt_sources:
  - source: "ppa:saltstack/salt"

packages:
  - salt-minion
  - salt-common
  - salt-master
  - python-software-properties
  - software-properties-common
  - python-novaclient

To boot an instance from the terminal, you can use the “nova” command like this;

nova boot --image Ubuntu-14.04-Trusty --user-data /srv/cloudconfig.yml --key_name keyname --flavor subsonic --security-groups default salt

This assumes that you have the following available in your OpenStack dashboard:

  1. An SSH public key called “keyname” in your tenant
  2. A flavor called “subsonic” that has a predefined configuration of vCPU, vRAM, etc.
  3. A security group called “default”, you could use more than one by separating them by comas; e.g. default,foo,bar
  4. A text file in /srv/cloudconfig.yml in YAML format that holds your Cloud-Init (a.k.a. “userdata”) configuration.
  5. You have your nova configuration available (look in your cloud provider dashboard “Download OpenStack RC File” link in “Access & Security” and “API access”) and available in your server’s /etc/profile.d/ profile folder.
  6. You have “python-novaclient” (or its equivalent) installed

To test it out yourself, you could use the block I gave earlier and create a file in /srv/cloudconfig.yml and give the the nova command a try.

In this case, you might want to call the new VM “salt” as the default Salt stack configuration will try to communicate to it to make it its salt master. In this case, it’ll be itself.

The creation of the salt master could also contain a few git repositories to be cloned at the salt master creation time making your salt master as easily replaceable as any other components in your “cloud”.

A set of sample scripts I use to create a new salt master off of a few git repositories can be found in the following Gist

Read more

The following articles was found to be describing in more detail what I introduced in this article.

Quelques bouts de code pour automatiser le déploiement

Automate ALL THE THINGS!

Ce billet n’est qu’un simple «link dump» pour retrouver parmi plusieurs notes éparpillés. Je compte éventuellement publier la totalité de mon travail dans des projets publics sur GitHub une fois la boucle complétée. Le tout sans fournir les données privés, évidemment.

Faire le saut vers l’automatisation demande beaucoup de préparation et je prends le temps de publier ici quelques bouts de code que j’ai écrits pour compléter la tâche.

Au final, mon projet permettra de déployer un site qui s’appuie sur un cluster MariaDB, Memcached, une stack LAMP («prefork») lorsqu’on a pas le choix, une stack [HHVM/php5-fpm, Python, nodejs] app servers pour le reste servi par un frontend NGINX. Mes scripts vont déployer une série d’applications web avec toutes les dépendances qui les adaptent géré dans leur propre «git repo» parent. Dans mon cas, ce sera: WordPress, MediaWiki, Discourse, et quelques autres.

Requis

  • Instantiation à partir de commandes nova du terminal, crée une nouvelle VM mise à jour et son nom définit son rôle dans le réseau interne
  • Les VMs sont uniquement accessible par un Jump box (i.e. réseau interne seulement)
  • Un système regarde si un répertoire clone git à eu des changements sur la branche «master», lance un événement si c’est le cas
  • Chaque machine sont construites à partir d’une VM minimale. Dans ce cas-ci; Ubuntu 14.04 LTS
  • Système doit s’assurer que TOUTES les mises à jour sont appliqués régulièrement
  • Système doit s’assurer que ses services interne sont fonctionnels
  • Dans le cas d’une situation où une VM atteint le seuil critique OOM, la VM redémarre automatiquement
  • Le nom de la VM décrit son rôle, et les scripts d’installation installent les requis qui y sont affectés
  • Les configurations utilisent les détails (e.g. adresses IP privés et publiques) de chaque pool (e.g. redis, memcache, mariadb) et ajuste automatiquement les configurations dans chaque application
  • … etc.

Bouts de code

Billets inspirants sur le sujet

Processus de création d’une VM faisant partie d’un parc géré par Salt Stack

A mon emploi actuel je gère un parc de machines virtuelles (VMs) qui est automatisé par un système de gestion de configuration appelé Salt Stack.

Salt stack est un outil permettant de décrire quel est l’état désiré d’un serveur. Comme d’autres outils avec une utilité similaire, Salt Stack peut être utilisé dans des environnements hétérogènes sous GNU/Linux, Mac OS, et Windows, FreeBSD, etc.

L’environment que nous utilisons est un serveur avec des «blades». Chaque «blade» fournit les services créant un cluster OpenStack. Dans le futur, nous risquons d’avoir plus d’un fournisseur OpenStack. Pour automatiser comme nous l’aimons, nous utilisons grandement la ligne de commande avec le paquet python-novaclient.

Chaque machine virtuelle roule une version LTS («Long Term Support») de Ubuntu.

Absolument toutes les configurations sont appliqués via Salt Stack, la seule chose qui est fait manuellement en
ce moment est de créer la nouvelle instance, et de l’ajouter au «master» de Salt Stack.

Même là, ça risque de changer lorsque nous aurons déployé Salt Cloud.

Procédure

Mise à jour Mars 2015: Un nouvel article sur le même sujet a été écrit (en anglais) et illustre comment faire une nouvelle VM avec encore moins d’étapes

  1. Boot une nouvelle node avec Nova

    joe@master:~$ nova boot --image lucid-minion-template --flavor wpdn.large --key-name renoirb app6
    
  2. Donner un nom en fonction du type de serveur a déployer avec un numéro à la fin. Exemple: app6

    NOTE Dans mon cas, j’ai notamment: app, db, memcached, etc.

  3. Ajoute l’adresse floating dans /srv/pillar/nodes/init.sls comme les autres

    nodes:
      master:
        public:  ####IP PUBLIQUE CACHÉE####
        private: 10.0.0.1
    
      app1:
        public:  ####IP PUBLIQUE CACHÉE####
        private: 10.0.0.7
    
      memcache2:
        public:  ####IP PUBLIQUE CACHÉE####
        private: 10.0.0.4
    
      app5:
        public:  ####IP PUBLIQUE CACHÉE####
        private: 10.0.0.3
    
  4. Prend le fichier /home/ubuntu/runme de n’importe quel autre serveur et colle le dans la nouvelle machine. Puis execute (sudo /bin/bash runme)

  5. Ajouter une ligne dans le nouveau serveir dans /etc/salt/minion.d/master.conf

    id: app6
    

    … Voir les autres nodes

  6. Restart salt-minion

    ubuntu@node6:~$ sudo service salt-minion restart
    
  7. Ajoute la clée au master

    joe@master:~$ sudo salt-key -a app6
    

    … Le ‘-a foo’ est optionnel et tu peux lister Les nodes.

  8. Run state.highstate

    joe@master:~$ sudo salt app6 state.highstate
    
  9. Uploader le code via rsync dans la nouvelle app node, puis re-rouler state.highstate (certains scripts prennent pour aquis que le code est déjà déployé)

    joe@master:~$ sudo salt app6 state.sls code.node_app
    joe@master:~$ sudo salt app6 state.highstate
    

    Comme je disais, parfois, le premier state.highstate ne marche pas a cause du code pas déployé.

  10. Rafraichir les autorisations pour storage

    joe@master:~$ sudo salt 'storage*' state.highstate
    joe@master:~$ sudo salt 'monitor*' state.highstate
    
  11. Updater le hosts file de quelque nodes

    joe@master:~$ sudo salt 'app*' state.sls hosts
    joe@master:~$ sudo salt 'db*' state.sls hosts
    joe@master:~$ sudo salt 'memcache*' state.sls hosts
    

Project idea: Creating a home made OpenStack cluster for development purposes

Think about it. How about using spare computers to create a homemade OpenStack cluster for development.

We can do that from our cloud provider, or create a separate project or even use Wikimedia’s OpenStack infrastructure allowance for the project.

With such setup, one could work locally with his Salt stack (or Puppet, or Ansible) deployment schemes, try them, trash VMs, rebuild.

The beauty of it would be that it could be made in a fashion that would not even modify the computer running the VMs. The cluster member running OpenStack hypervisor would be installed seeded through net boot. Not booting from the network would revert the computer back as if it never been used.

Here is what I think would require to make this happen.

Limitations

  • Not use Computer/Laptop local hard drive
  • Rely only on net boot

Material

  • 1..n Computers/laptop supporting netboot
  • 1 Storage device supporting one or more storage protocol (nfs, samba, sshfs)

Hardware requirements

  • 1 VM providing tftp, dhcp, dns to serve as net boot server that should run outide of the cluster (“Networking node”)
  • 1 VM image of OpenStack controller (“OpS controller”)
  • 1 LiveCD+persistent image with OpenStack preinstalled, configured to use storage device credentials as it’s root filesystem (“OpS Hypervisor”)

Distribution choice factors

  • Networking node can be the smallest Linux possible, on a RaspberryPI, or a modified Router or Network Attached storage device?
  • OpS Hypervisor to be among the supported OpenStack distributions (I think a Ubuntu precise 12.04 LTS or a variant such as Puppy Linux might work too)

To be continued…

I will keep you posted whenever possible on the outcome of my research.

Did you ever do this in your infra. Leave a comment.

What is Cloud computing when it is related to web application

During the discussion, the contributor persisted on knowing what would be considered and thresholds to use some kind of push-button-scaling.

Knowing his context, a unzipped install CMS with a buch of plugins I felt the urge to explain that there is not always need to get a bigger server capacity. Here is an overview of what I mean when I talk about cloud computing and continuous integration.

The E-Mail

Let’s talk about cloud! 

I mean in the web application hosting realm. Not the storage (Google Drive, Dropbox) or software as a service (Salesforce, Basecamp).

Let’s talk about a use case before and my own experience.

My former company Evocatio Solutions technologiques manage a pretty large site at the domain uda.ca.

The use-case on my recent experience

This is a complete business management web application that manages an union who represents french speaking artists in north america (mostly residents of Canada). We built a complete web application that manages many aspects an artist needs to represent themselves and be found. A big part of it is a 140 tables worth of artist description listing details as small a hair length and types of musical instruments to voice tones. It also manages renewal, communication with agencies, portfolios, and management of contracts with managers and more.

Not to forget the very heavy databases queries we generate to search, for example: <example>An asian woman with white hair playing yuku lélé who can pilot helicopter AND ride motorcycle …</example>

Yes. Database queries get very big, very quickly. Not only in the search engine I described, but through all the features.

That, to my opinion, is heavy. Also considering that that Artist’s Union has several thousand members.

This information is on top of my head, please do not take this into real numbers, I did not look the latest deployment needs.  But for the server side, it only uses a simple Virtual machine with 4Gb of RAM give or take.

That is my point about expanding hosting without optimizing stuff around.

What your web application has to consider then

Amazon and other Cloud service is about mostly about automated server deployment.

But the powerful offering of “scale tour application” with computing cubes that automatically scales requires more than just nodes.

It requires the code (here again) to support:

  • multiple databases hosts and types support (Cassandra, Solr, MySQL) specialized for the type of data to store
  • User upload files replication
  • Database/Keystore (CouchDB, Mongo)

All spanable on multiple hosts by a mere change of one configuration file.

The code itself should:

  • Be deployable by a simple phing/ant/nant task
  • Hosted on a NAS mount that you could create an other machine and use when time of computing need happens

All this (for some parts) is what is called Continuous integration (Wikipedia) some deployment strategies (also here and this blog post too), and most of the time. It’s not just the continuity and automation that matters, but the underlying deployment mechanism can be provided by third parties, like Heroku and many others.

Lancement de l’annee 2011 pour la conférence ConFoo

ConFoo.ca Web Techno Conférence

Depuis le debut de l’ete je me suis joint à l’organisation de la conférence ConFoo qui couvre les sujets technique comme les langages PHP, .Net, Python et les standards du web. Ce week-end aura lieu leur traditionnel BBQ de lancement d’année.

Detail de l’invitation officielle

Je duplique le contenu provenant de l’annonce offieielle de W3QC et de l’équipe d’organisation.

Le BBQ aura lieu le 11 septembre à partir de 13h.

Au menu: hot-dogs, chips, salades, boissons gazeuses, biscuits et beaucoup de plaisir. Dans ce même esprit, l’équipe de Montréal-Python apportera un keg de Charmeuse de Serpents. Une bière India Pale Ale spécialement brassée pour cet événement.

Afin de garantir le nombre adéquat de portions, veuillez confirmer votre présence en envoyant un email à board@confoo.ca. Noubliez pas de mentionner le nombre de personne qui vous accompagnera.

Notez le numéro 1-888-679-8466 option 0 au cas où vous ne trouverez pas le groupe sur les lieux.

Détails de l’évènement

  • Quand: 2010-09-11 à 13h
  • Où: parc Mont-Royal, près de la maison Smith (suivez les affiches)
  • Pour qui: les développeurs, les acteurs du Web et leurs familles
  • Tarif: gratuit!

Au plaisir de vous y voir.

L’équipe de ConFoo.