This blog post documents the process I went through to get Selenium running on a headless Linux server in my lab environment. I find Selenium extremely useful when regression testing web site upgrades and major changes. At the end of this post, I have included the code I use for the automated testing of a base (vanilla) Joomla installation.

Using the Selenium IDE extension in Chrome

 
First, I recorded individual tests in Chrome using  Selenium IDE extension
Then I added Python driver for Selenium, as pip was already installed:
pip install -U selenium

Then install pytest
sudo apt install python-logilab-common
sudo apt-get install pytest
 
I now needed to install Browser specific files:
Chrome: https://sites.google.com/a/chromium.org/chromedriver/downloads

 
So to install Firefox:
sudo add-apt-repository ppa:ubuntu-mozilla-security/ppa
sudo apt-get update
sudo apt-get install firefox
 
And to install ChromeDriver:
cd /usr/local/bin
wget https://chromedriver.storage.googleapis.com/2.35/chromedriver_linux64.zip
unzip chromedriver_linux64.zip
 
However, ChromeDriver still needs Chrome to be installed, so:
sudo nano /etc/apt/sources.list
 
Add at the bottom of the file
deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main
sudo wget https://dl.google.com/linux/linux_signing_key.pub
sudo apt-key add linux_signing_key.pub
which gives OK
 
sudo apt update
sudo apt install google-chrome-stable
Run Selenium Server, which I have in the below Bash script:
#!/bin/bash
# Runs the Selenium Server so run in background mode
java -Xms40m -Xmx256m -jar /home/pgroom/selenium-server-standalone-3.141.59.jar &
 
Then export the Selenium IDE tests that I created at the beginning of this post, as Python files and copy them to the server. Put them in Joomla home/tests/system and make them owned by www-data:www-data.
Check they run ok via
pytest <name of test.py>
 
This gave no errors but also does not do any tests. This is because this is a Ubuntu Server without a GUI i.e. headless so it cannot access a browser other than via a virtual X Server, as per below:
Install X-Virtual Frame Buffer (to mimic a GUI)
sudo apt-get install xvfb
pip install PyVirtual Display

The :40 is a display number so does not need to be 40 per se.
sudo nohup Xvfb :40 -ac &
export DISPLAY=:40
Check by doing
env | grep DISPLAY
 
There are two options for running the tests in headless mode. One is using ChromeOptions (which is commented out below) and the other is using PyVirtualDisplay. Thank you to Oren Nahum (https://blog.testproject.io/2018/02/20/chrome-headless-selenium-python-linux-servers/) for the PyVirtualDisplay code that worked a treat.
 
For clarity, running the Selenium Server, X Virtual Frame Buffer and setting the DISPLAY are only necessary if a remote website is being tested. If the website is local to the machine then these are not necessary.
 

Final script for the default Joomla installation

 
import pytest
import time
import json
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
 
# Code for ChromeOptions as an alternative to pyvirtualdisplay
# chrome_options = webdriver.ChromeOptions()
# chrome_options.add_argument('--headless')
# chrome_options.add_argument('--no-sandbox') # required when running as root user. otherwise you would get no sandbox errors.
# driver = webdriver.Chrome(driver_path='/usr/local/bin/chromedriver', chrome_options=chrome_options, service_args=['--verbose', '--log-path=/tmp/chromedriver.log'])
 
# Code for pyvirtualdisplay as an alternative to ChromeOptions
from pyvirtualdisplay import Display
display = Display(visible=0, size=(1024, 768))
display.start()
 
class test_uatchromedesktoptests():
# Full page web tests
   print("DESKTOP WEB TESTS")
   print("")
 
# Check to see Getting Started is still on the Home Page
   driver = webdriver.Chrome(executable_path='/usr/local/bin/chromedriver', service_args=['--verbose', '--log-path=/tmp/chromedriver.log'])
   driver.get('<URL of test web site>')
   print("Test 1 - UAT Chrome Desktop text exist")
   driver.set_window_size(1294, 694)
   assert ("Getting Started" in driver.page_source)
   print("Passed - found Getting Started on the page")
   print("")
 
# Check can find and click on Home button.
   print("Test 2 - UAT Chrome Desktop Home button exist")
   menuhome = driver.find_element_by_css_selector("body.site > div#top > div.container > nav.navigation > div.nav-collapse > ul.nav > li.item-101 > a")
   print(menuhome.text)
   print("Passed - found Menu -> Home on the page")
   print("")
 
# Check can find and click on Joomla button
   print("Test 3 - UAT Chrome Desktop Joomla button exist")
   joomlabtn = driver.find_element_by_css_selector("body.site > div#top > div.container > div.row-fluid > main#content > div.item-page > ul.tags > li.tag-2 > a.label")
   print(joomlabtn.text)
   print("Passed - found Joomla button on the page")
   print("")
 
# Check can find and click on Joomla link under Popular Tags
   print("Test 4 - UAT Chrome Desktop Joomla under Popular Tags exist")
   populartags = driver.find_element_by_css_selector("body.site > div#top > div.container > div.row-fluid > div#aside > div.well > div.tagspopular > ul > li > a")
   print(populartags.text)
   print("Passed - found Popular Tags -> Joomla on the page")
   print("")
 
# Check can find and click on Getting Started under Latest Articles
   print("Test 5 - UAT Chrome Desktop Getting Started under Latest Articles exist")
   gettingstarted = driver.find_element_by_css_selector("body.site > div#top > div.container > div.row-fluid > div#aside > div.well > ul.latestnews > li > a > span")
   print(gettingstarted.text)
   print("Passed - found Latest Articles -> Getting Started on the page")
   print("")
 
# Check can enter "CMS" in Search box and get 0 results as a result
   print("Test 6 - UAT Chrome Desktop 0 results for CMS in Search Box exist")
   searchentry = driver.find_element_by_css_selector("body.site > div#top > div.container > header.header > div.header-inner > div.header-search > div.search > form.form-inline > input#mod-search-searchword93.inputbox")
   searchentry.send_keys('CMS')
   searchentry.send_keys(Keys.ENTER)
   assert ("results" in driver.page_source)
   searchresults = driver.find_element_by_css_selector("body.site > div#top > div.container > div.row-fluid > main#content > div.search > form#searchForm > div.searchintro > p > strong > span.badge") 
   print(searchresults.text)
   print("Passed - found 0 results for CMS in Search Box on the page")
   print("")
 
# Check can login in using Username and Password
# Then check get an User Menu with links to "Your Profile", "Submit an Article", "Site Administrator", "Template Setttings" and "Site Settings"
   print("Test 7 - UAT Chrome Desktop Logged in and get an User Menu with links")
   username = driver.find_element_by_id("modlgn-username")
   password = driver.find_element_by_id("modlgn-passwd")
   username.send_keys("username") # CHANGE THIS TO ACTUAL USERNAME
   password.send_keys("password") # CHANGE THIS TO ACTUAL PASSWORD
   driver.find_element_by_name("Submit").click()
   assert ("Your Profile" in driver.page_source)
   assert ("Submit an Article" in driver.page_source)
   assert ("Site Administrator" in driver.page_source)
   assert ("Template Settings" in driver.page_source)
   assert ("Site Settings" in driver.page_source)
   print("Passed - Logged in and found Your Profile etc on the page")
   print("")
 
# Then check that the name in the Name box is "Super User"
   print("Test 8 - UAT Chrome Desktop Logged in -> User Menu -> Your Profile -> Name -> Super User")
   yourprofile = driver.find_element_by_css_selector("body.site > div#top > div.container > div.row-fluid > div#aside > div.well > ul.nav > li.item-102 > a").click()
   assert ("Super User" in driver.page_source)
   print("Passed - Selected User Menu -> Your Profile -> Name results in Super User")
   print("")
 
# Finally check can log out and the User Menu disappears
   print("Test 9 - UAT Chrome Desktop Logged Out -> back to Login")
   logout = driver.find_element_by_css_selector("body.site > div#top > div.container > div.row-fluid > div#aside > div.well > form#login-form > div.logout-button > input.btn").click()
   assert ("Login Form" in driver.page_source)
   print("Passed - Logged out back to Login prompt")
   print("")
   driver.quit()
class test_uatchromemobiletests():
 
# Mobile web tests
   print("MOBILE WEB TESTS")
   print("")
 
# Check to see Getting Started is still on the Home Page
   driver = webdriver.Chrome(executable_path='/usr/local/bin/chromedriver', service_args=['--verbose', '--log-path=/tmp/chromedriver.log'])
   driver.get('<URL of test web site>')
   print("Test 11 - UAT Chrome Mobile text exist")
   driver.set_window_size(514, 662)
   assert ("Getting Started" in driver.page_source)
   print("Passed - found Getting Started on the page")
   print("")
 
# Check can find and click on Home button.
   print("Test 12 - UAT Chrome Mobile Home button exist")
   menuhome = driver.find_element_by_css_selector("body.site > div#top > div.container > nav.navigation > div.nav-collapse > ul.nav > li.item-101 > a")
   print(menuhome.text)
   print("Passed - found Menu -> Home on the page")
   print("")
 
# Check can find and click on Joomla button
   print("Test 13 - UAT Chrome Mobile Joomla button exist")
   joomlabtn = driver.find_element_by_css_selector("body.site > div#top > div.container > div.row-fluid > main#content > div.item-page > ul.tags > li.tag-2 > a.label")
   print(joomlabtn.text)
   print("Passed - found Joomla button on the page")
   print("")
 
# Check can find and click on Joomla link under Popular Tags
   print("Test 14 - UAT Chrome Mobile Joomla under Popular Tags exist")
   populartags = driver.find_element_by_css_selector("body.site > div#top > div.container > div.row-fluid > div#aside > div.well > div.tagspopular > ul > li > a")
   print(populartags.text)
   print("Passed - found Popular Tags -> Joomla on the page")
   print("")
 
# Check can find and click on Getting Started under Latest Articles
   print("Test 15 - UAT Chrome Mobile Getting Started under Latest Articles exist")
   gettingstarted = driver.find_element_by_css_selector("body.site > div#top > div.container > div.row-fluid > div#aside > div.well > ul.latestnews > li > a > span")
   print(gettingstarted.text)
   print("Passed - found Latest Articles -> Getting Started on the page")
   print("")
 
# Check can enter "CMS" in Search box and get 0 results as a result
   print("Test 16 - UAT Chrome Mobile 0 results for CMS in Search Box exist")
   searchentry = driver.find_element_by_css_selector("body.site > div#top > div.container > header.header > div.header-inner > div.header-search > div.search > form.form-inline > input#mod-search-searchword93.inputbox")
   searchentry.send_keys('CMS')
   searchentry.send_keys(Keys.ENTER)
   assert ("results" in driver.page_source)
   searchresults = driver.find_element_by_css_selector("body.site > div#top > div.container > div.row-fluid > main#content > div.search > form#searchForm > div.searchintro > p > strong > span.badge") 
   print(searchresults.text)
   print("Passed - found 0 results for CMS in Search Box on the page")
   print("")
 
# Check can login in using Username and Password
# Then check get an User Menu with links to "Your Profile", "Submit an Article", "Site Administrator", "Template Setttings" and "Site Settings"
   print("Test 17 - UAT Chrome Mobile Logged in and get an User Menu with links")
   username = driver.find_element_by_id("modlgn-username")
   password = driver.find_element_by_id("modlgn-passwd")
   username.send_keys("username") # CHANGE THIS TO ACTUAL USERNAME
   password.send_keys("password") # CHANGE THIS TO ACTUAL PASSWORD
   driver.find_element_by_name("Submit").click()
   assert ("Your Profile" in driver.page_source)
   assert ("Submit an Article" in driver.page_source)
   assert ("Site Administrator" in driver.page_source)
   assert ("Template Settings" in driver.page_source)
   assert ("Site Settings" in driver.page_source)
   print("Passed - Logged in and found Your Profile etc on the page")
   print("")
 
# Then check that the name in the Name box is "Super User"
   print("Test 18 - UAT Chrome Mobile Logged in -> User Menu -> Your Profile -> Name -> Super User")
   yourprofile = driver.find_element_by_css_selector("body.site > div#top > div.container > div.row-fluid > div#aside > div.well > ul.nav > li.item-102 > a").click()
   assert ("Super User" in driver.page_source)
   print("Passed - Selected User Menu -> Your Profile -> Name results in Super User")
   print("")
 
# Finally check can log out and the User Menu disappears
   print("Test 19 - UAT Chrome Mobile Logged Out -> back to Login")
   logout = driver.find_element_by_css_selector("body.site > div#top > div.container > div.row-fluid > div#aside > div.well > form#login-form > div.logout-button > input.btn").click()
   assert ("Login Form" in driver.page_source)
   print("Passed - Logged out back to Login prompt")
   print("")
   driver.quit()
 
You might be wondering why the same tests were run for both the desktop and mobile display sizes. In practice, I have found it is best to split the two types as the layouts are usually significantly different. This means that the same css elements are often referenced in different ways between the desktop and mobile renditions of the site.
 
Well that covers everything I did to get Selenium up and running with Joomla. I have done posts on Docker, Ansible and Php & Curl. I hope that you found this article useful and please feel free to contact me if you have any questions. 
 
Thanks
 
Pete

How could Peter help?

As an IT consultant and technology advisor, Peter is often used for his significant digital/online transformation, troubleshooting, assessor, due diligence and turnaround experience, particularly if the word "impossible" has been used. With a reputation as a fixer, firefighter and troubleshooter, Peter takes on the most challenging of situations and brings to bear his extensive IT, digital & technology turnaround experience, coupled with his skills as a CEO and Board level advisor. 

Available to help on a full time, part time, ad hoc or fractional basis as necessary. Equally comfortable working onsite or remote, depending on the challenge.

Why not get in touch for a free, no obligation, completely "off the record" chat ...?

 

Specialties

Peter Groom has significant experience in delivering and scaling online/digital services, as well as system migrations and upgrades. Well versed in E-Commerce platforms, both off the shelf and bespoke and using public, private and hybrid clouds. Highly experienced in delivering business value as a technology consultant, IT assessor and digital advisor, where IT and technology are seen as an enabler (think of it as "Lego") and often in high pressure turnaround environments.

Delivers innovation by using proven technology in one sector and applying it to a business challenge in another sector. Has an excellent track record as an IT consultant, firefighter and troubleshooter, of delivering "against the odds", initiatives whilst also bringing the organisation along on the journey. Extensive troubleshooting experience across many global markets, where his assessment and due diligence expertise prove essential, together with his CEO & Board level advisor skills.

His focus on pragmatic outcomes has delivered significant growth for his clients, across a multitude of sectors and continues to do so. In fact, it is this focus on practical actions, particularly from IT Assessments and Due Diligence, together with his guarantee that he will (if the client wishes) deliver those results for the client, that set him apart from many others. 

 

Contact Peter

Email Peter Groom, IT troubleshooter & digital consultant [email protected]

 

Call Peter Groom, IT firefighter & online firefighter and fixer +44 (0)7710 745360

Contact Pete
1000 characters left