Tag Archives: Banana Pi

Moving to a larger cluster

I’ve been building a larger version of my Banana Pi cluster. It was originally built with just four nodes, but I’ve extended it to 8. There are
now four database servers running MySQL which are synchronized using master/slave replication. There are four web servers with Nginx and uWSGI
running Pyplate.

I plan on adding another four Banana Pis as web servers. Pages are cached on the web servers, so most pages are served without needing to access
the database. There isn’t much load on the database servers, so it shouldn’t be a problem to have more web servers than database servers.

The web servers are synchronized using rsync. I’ve set up ssh keys so that I can use rsync over ssh without having to enter passwords. The web
root directory and the Pyplate directory are synchronized from a master web server to the workers in the web server cluster.

I’ve moved pyplate.com to the new cluster, and I’m working on new sites that will also be hosted on the cluster.

I’m using Ganglia to monitor both clusters. It’s almost set up the way I want, except the master web server doesn’t appear in the Ganglia UI.
I’ll check the settings in gmond.conf on the node I’m having problems with.

Advertisements

Running several sites on a Banana Pi server

I’ve been doing some work on Pyplate CMS.  I’ve upgraded it so that I can run several sites on a single server, and I’ve also added support for MySQL to Pyplate.  I’m calling the new version Pyplate Multi-Site.

Now I can create a very simple mass blogging system on my Banana Pi cluster by installing Pyplate Multi-Site on each server in the cluster and make sure that they are synchronized.  This is an improvement on the Raspberry Pi cluster which can only serve one site.

I haven’t released the updated code yet.  I need to do more testing and update the Pyplate installation scripts.

So far I have only tested Pyplate Multi-site with Nginx.  If I want to use it with Apache, I will need to use mod vhosts in order to set a different web root folder for each site.  I haven’t tested it yet, but I will over the next few weeks.

In the short term I need to keep working on the Banana Pi cluster and get the upgraded version of it online so that I can start building some new sites.

GlusterFS and MySQL clusters running on Banana Pi servers

I’ve set up a four node MySQL cluster using Banana Pi servers.  There’s one master and three slaves replicating a MySQL database.

4 Banana Pi servers

4 node Banana Pi cluster

The first time I tried this I set up a small two node cluster and set up replication by typing commands at the MySQL shell prompt.  I didn’t want to do that again when I set up a 4 node cluster, so I wrote some Python scripts to do the work for me.  You can check them out at https://github.com/Pyplate/db-cluster-utils.

For detailed information on how I set this up, check out this page:http://banoffeepiserver.com/mysql/automate-mysql-replication-on-four-nodes.html.

I’ve also built a cluster of Banana Pi servers for use with the GlusterFS file distributed system.  First I built a stack of hard disks:

4 SATA hard disks

4 SATA hard disks

I connected a 4 node Banana Pi cluster to the stack, and installed GlusterFS on each Banana Pi.  Data is replicated on two of the disks, so if one disk breaks, no data will be lost.  You can read about how I set this up at http://banoffeepiserver.com/glusterfs/setting-up-glusterfs-on-four-banana-pi-servers.html

4 Banana Pis and 4 hard disks

4 node Gluster volume with SATA disks running on Banana Pi servers

Once I’ve finished testing these clusters, I can integrate them with my cluster of Banana Pi web servers.

My latest site, BanoffeePiServer.com, has been running smoothly since I set it up. I’m really pleased with its performance. At the moment it’s hosted on a small cluster of Banana Pi servers. I now have 10 more Banana Pis, so I’m planning on building a more sophisticated cluster.

I’ve been working on adding support for MySQL to Pyplate. This will mean I can set up a cluster of database servers, and offload the database from the nodes running the Nginx.

I have got as far as setting up a MySQL database and importing data into it. I’ve modified Pyplate to remotely access the MySQL database instead of the local SQLite database.

Next, I need to set up three more database nodes, and set up database replication. Then when I make changes to one of the databases, the changes will automatically be flushed through to the other database nodes.

I’ve got a lot more testing to do, but eventually I will release an update of Pyplate with added support for MySQL.

Once I’ve installed the MySQL cluster, I’ll look into setting up four more nodes running the Gluster distributed file system.

ARM powered server farm

My Banana Pi site has been up for a few days now, and I’m very happy with its performance.  I’ve put my clusters together to form a nice neat stack:

ARM server farm

ARM server farm

The Banana Pi servers are in the rack on the left.  The other two racks contain Raspberry Pi servers.

Banana Pi and Raspberry Pi server farm

Banana Pi and Raspberry Pi server farm

You can read about how I set up the Banana Pi cluster here: http://banoffeepiserver.com/banana-pi-server-cluster/.

Banana Pi server cluster

Banana Pi server cluster

Banana Pi server cluster

I built a Raspberry Pi cluster a few months ago, and it’s been very reliable. Performance has been surprisingly good, and the cluster has shown to handle a reasonable amount of traffic.

I recently got my hands on four Banana Pis, a Raspberry Pi clone with a dual core processor and gigabit ethernet.  I’ve built another cluster and got it on line at BanoffeePiServer.com.  This cluster has half the number of nodes in the RPi cluster, but it’s much more powerful.

I’ve used a newer version of Pyplate CMS which is much more efficient, and I’ve used a much more efficient mechanism to synchronize the servers.  Pyplate uses an SQLite database, so there’s no need for database replication.  The entire CMS can be copied to each node with rsync.

This page explains how I built the Banana Pi cluster.

A new version of Pyplate CMS is now available

It’s a long time since I posted here. I’ve been working on Pyplate, my Python CMS. Over the last few months I’ve made a lot of changes to the UI. Most site administration tasks can now be done via the Pyplate UI.

I’ve modified Pyplate to use the WSGI server interface instead of CGI. With CGI, the Python interpreter has to be loaded everytime a script is executed. Using WSGI means that a Python interpreter is kept in memory between requests. When a script needs to be executed it can run immediately because the interpreter has already been loaded. This has no effect on page load times of static pages, but dynamic pages are served much more quickly. The difference is huge.

There are different implementations of WSGI for different servers. In order to use WSGI with Apache, libapache2-mod-wsgi must be installed. This ensures that a python interpreter is loaded into each Apache thread. The downside of this is that each thread takes up more memory.

With Nginx, people often use set up a uWSGI server.  Nginx must be configured to pass requests for scripted pages to the uWSGI server. The uWSGI server executes a script, and returns the result to Nginx, which then passes the page back to the user who requested it.

I’ve also implemented a very simple webserver based on the WSGI class in the Python library. This server can handles static pages. If a requested file can’t be found, the server hands the request to Pyplate.  Pyplate generates the page, and returns it to the server.

The Python server is not very fast. It’s slower than Apache, and enabling page caching doesn’t make much difference. On the upside, it is very easy to set up.

I’ve added some security patches to Pyplate. A security token is embedded in each form generated by Pyplate. When a user clicks on the submit button, the security token is sent to the server with the POST request, and validated. I’ve also made some changes to the cookie code to make it harder to steal user’s cookies.

User information is now stored more securely. A salt is used to encrypt each user’s password, and a different salt is generated for each user.

I think Pyplate has come a very long way. It’s now a much more complete, more usable CMS. It’s still pretty simple, and small enough to run quickly.

What’s next? I’m working on a site built on a Banana Pi cluster. I’m building it using Pyplate, and it’s going to be based on my Raspberry Pi site. I may add a few more articles to my Raspberry Pi site, but eventually I plan to get back to working on Pyplate.