Jekyll2023-09-01T17:08:38+03:00https://jperala.fi/feed.xmlTest, Develop & AutomateTest, develop, automateMake side projects that matter2021-01-11T14:41:00+02:002021-01-11T14:41:00+02:00https://jperala.fi/2021/01/11/make-side-projects-that-matter<p>For me learning tech and new skill is not just for having 9-5 work and paying the bills. I have always liked to have side projects. As far as I can remember I have always tinkered something on the side of my day job. When it was learning new language or framework like React, testing tool like Cypress or dabbling with Raspberry Pi, Nano or some other gadget of interest.</p>
<h1 id="tech-or-content-first">Tech or Content First?</h1>
<p>For me the side projects have always started tech first, from the interest to learn some new tool or technology. That is totally fine for learning and applying the learned skills in real world example. But it has also meant that the developed side projects have been dispensable. I mean like fire and forget type of things that - once finished - have been doomed to die. There has been no motivation to keep the projects up and running afterwards.</p>
<p>This was before my last side project that I started about an year a go. I wanted to make a larger side project where I could really learn and apply my skills from every aspect of developing an online product and presence.</p>
<p><a href="img/2021-01-11/blog.jpg"><img src="/img/2021-01-11/blog.jpg" alt="Ranteessa blog" /></a></p>
<p>To keep my interest on the project I chose the topic to be inline with my other hobbies. As I had been watch enthusiast for several years by then I deciced to start an wristwatch media site dedicated for watch enthusiasts and others interested about wristwatches.</p>
<p>I named the project as <em>Ranteessa</em> (‘<em>On the Wrist</em>’ in English) and it consist of <a href="https://ranteessa.fi" target="_blank">ranteessa.fi</a> blog with associated <a href="https://youtube.com/c/Ranteessa" target="_blank">YouTube</a>, <a href="https://www.instagram.com/ranteessa/" target="_blank">Instagram</a> and <a href="https://www.facebook.com/ranteessafi/" target="_blank">Facebook</a> accounts. Later I added also <a href="https://twitter.com/ranteessafi" target="_blank">Twitter</a> account although it gets updated less regularly.</p>
<h1 id="the-value-and-victory">The Value and Victory</h1>
<p>And, oh boy, has this paid off. Instead of going tech firsts to do some random project working for topic relevant to you is SO MUCH more interesting.</p>
<p>Working with project expanded across multiple platforms and with real subscribers and followers you get the opportunity to learn so much more. Basically everything from developing and hosting a website to analytics, social media and affiliate marketing. And not just the tech stuff, biggest learning curve has come with content creation including things like video production, editing, blogging, presentation skills and much more.</p>
<p>In grand scheme this is still just a miniscule project, but for me it has been a huge victory. As of today blog is getting steadily over thousand unique visitors monthly and both Instagram and YouTube channels have gained around 500 followers. Not bad for a nerdy Finnish speaking channel talking about wristwatches. But by far the biggest value of the project has been learning new skills and connecting with other watch enthusiasts around the Finland.</p>
<p>Suprisingly the Ranteessa project has also moved bit of beyond being just a hobby project. It has also started gaining attraction for collaborations with commercial watch companies and stakeholders. Only time will tell what will this evolve in the future.</p>
<h1 id="passion-rules-the-game">Passion Rules The Game</h1>
<p>When you think about starting your next side project, pick a topic that you are really interested about. Something close to your heart and other hobbies. It will pay off, I promise.</p>
<p>Ps. If you are interested on wristwatches, please feel free to take a look of Ranteessa on <a href="https://www.instagram.com/ranteessa/" target="_blank">Instagram</a>, <a href="https://youtube.com/c/Ranteessa" target="_blank">YouTube</a>, and <a href="https://ranteessa.fi" target="_blank">ranteessa.fi</a> blog. If you like it, please consider also subscribing and following to help me to beat the evil algorithm.</p>jperalaFor me learning tech and new skill is not just for having 9-5 work and paying the bills. I have always liked to have side projects. As far as I can remember I have always tinkered something on the side of my day job. When it was learning new language or framework like React, testing tool like Cypress or dabbling with Raspberry Pi, Nano or some other gadget of interest. Tech or Content First? For me the side projects have always started tech first, from the interest to learn some new tool or technology. That is totally fine for learning and applying the learned skills in real world example. But it has also meant that the developed side projects have been dispensable. I mean like fire and forget type of things that - once finished - have been doomed to die. There has been no motivation to keep the projects up and running afterwards. This was before my last side project that I started about an year a go. I wanted to make a larger side project where I could really learn and apply my skills from every aspect of developing an online product and presence. To keep my interest on the project I chose the topic to be inline with my other hobbies. As I had been watch enthusiast for several years by then I deciced to start an wristwatch media site dedicated for watch enthusiasts and others interested about wristwatches. I named the project as Ranteessa (‘On the Wrist’ in English) and it consist of ranteessa.fi blog with associated YouTube, Instagram and Facebook accounts. Later I added also Twitter account although it gets updated less regularly. The Value and Victory And, oh boy, has this paid off. Instead of going tech firsts to do some random project working for topic relevant to you is SO MUCH more interesting. Working with project expanded across multiple platforms and with real subscribers and followers you get the opportunity to learn so much more. Basically everything from developing and hosting a website to analytics, social media and affiliate marketing. And not just the tech stuff, biggest learning curve has come with content creation including things like video production, editing, blogging, presentation skills and much more. In grand scheme this is still just a miniscule project, but for me it has been a huge victory. As of today blog is getting steadily over thousand unique visitors monthly and both Instagram and YouTube channels have gained around 500 followers. Not bad for a nerdy Finnish speaking channel talking about wristwatches. But by far the biggest value of the project has been learning new skills and connecting with other watch enthusiasts around the Finland. Suprisingly the Ranteessa project has also moved bit of beyond being just a hobby project. It has also started gaining attraction for collaborations with commercial watch companies and stakeholders. Only time will tell what will this evolve in the future. Passion Rules The Game When you think about starting your next side project, pick a topic that you are really interested about. Something close to your heart and other hobbies. It will pay off, I promise. Ps. If you are interested on wristwatches, please feel free to take a look of Ranteessa on Instagram, YouTube, and ranteessa.fi blog. If you like it, please consider also subscribing and following to help me to beat the evil algorithm.How to run Robot Framework from command line?2020-03-25T17:23:00+02:002020-03-26T17:16:00+02:00https://jperala.fi/2020/03/25/how-to-run-robot-framework-tests-from-command-line<p>Yesterday a colleague learning test automation forgot the syntax for running Robot Framework test cases from command-line. Although full command-line syntax is described on the <a href="https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html" target="_blank">Robot Framework User Guide</a> it is rather long and tedious document to dive into.</p>
<p>This post aims to provide quick introduction for executing single, set or all Robot Framework test cases in a project.</p>
<h1 id="basic-syntax">Basic syntax</h1>
<p>The basic syntax for executing Robot Framework tests from command line is:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh"> <span class="c"># basic syntax
</span>
robot <span class="o">[</span>options] robot_files
<span class="c"># example with options
</span>
robot <span class="nt">-v</span> URL:http://example.com example.robot</code></pre></figure>
<p>For full list of command-line options see <code class="highlighter-rouge">robot --help</code> or <code class="highlighter-rouge">robot -h</code> option.</p>
<h1 id="execute-all-test-cases-in-folders">Execute all test cases in folder(s)</h1>
<p>To run all robot tests in the folder (including subfolders) use <code class="highlighter-rouge">.</code></p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh"> <span class="c"># execute all tests in all robot files in current folder and subfolders
</span>
robot .</code></pre></figure>
<h1 id="execute-all-test-cases-in-single-file">Execute all test cases in single file</h1>
<p>To run all robot tests in single robot file:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh"> <span class="c"># execute all tests in single robot file in current folder
</span>
robot example.robot
<span class="c"># execute all tests in single robot file in subfolder
</span>
robot path/to/example.robot</code></pre></figure>
<h1 id="execute-test-cases-by-test-name">Execute test cases by test name</h1>
<p>To run test cases with specific test name use <code class="highlighter-rouge">--test</code> or <code class="highlighter-rouge">-t</code> option:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh"> <span class="c"># execute test cases with name "Example" in any file.
</span>
robot <span class="nt">--test</span> Example <span class="nb">.</span>
<span class="c"># execute test cases with name "Example" in specific file.
</span>
robot <span class="nt">--test</span> Example example.robot</code></pre></figure>
<p>Also partial test names and patterns can be used with <code class="highlighter-rouge">--test</code> option, see <a href="#using-partial-names-and-filter-patterns">Using partial names and filter patterns</a>.</p>
<h1 id="execute-test-cases-by-tags">Execute test cases by tags</h1>
<p>Test cases and suites annotated with tags (using <code class="highlighter-rouge">[Tags]</code> or <code class="highlighter-rouge">Force Tags</code> syntax) can be executed by selecting tags to be <em>included</em> or <em>excluded</em>.</p>
<p>Example test suite:</p>
<figure class="highlight"><pre><code class="language-robotframework" data-lang="robotframework"> *** Settings ***
Force Tags suite
*** Test Cases ***
Test One
[Tags] one
No operation
Test Two
[Tags] two
No operation</code></pre></figure>
<p>Also partial tag names and patterns can be used with <code class="highlighter-rouge">--include</code> and <code class="highlighter-rouge">--exclude</code> option, see <a href="#using-partial-names-and-filter-patterns">Using partial names and filter patterns</a>.</p>
<h2 id="include-test-cases-by-tag">Include test cases by tag</h2>
<p>To run test cases with specific tag name included use <code class="highlighter-rouge">--include</code> or <code class="highlighter-rouge">-i</code> option:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh"> <span class="c"># execute test cases with tag "one" in any file.
</span>
robot <span class="nt">--include</span> one <span class="nb">.</span>
<span class="c"># execute test cases with tags "one" and "two" in any file.
</span>
robot <span class="nt">--include</span> oneANDtwo <span class="nb">.</span>
<span class="c"># execute test cases with tag "one" or "two" in any file.
</span>
robot <span class="nt">--include</span> oneORtwo <span class="nb">.</span>
<span class="c"># execute test cases with tag "one" but without tag "two" in any file.
</span>
robot <span class="nt">--include</span> oneNOTtwo <span class="nb">.</span> </code></pre></figure>
<h2 id="exclude-test-cases-by-tag">Exclude test cases by tag</h2>
<p>To run test cases with specific tag name excluded use <code class="highlighter-rouge">--exclude</code> or <code class="highlighter-rouge">-e</code> option:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh"> <span class="c"># execute test cases without tag "two" in any file.
</span>
robot <span class="nt">--exclude</span> two .</code></pre></figure>
<h1 id="execute-test-cases-by-suite-name">Execute test cases by suite name</h1>
<p>In <em>Robot Framework</em> test folders and <code class="highlighter-rouge">.robot</code> files are considered as test suites.</p>
<p>Example suite structure:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh"> - +-+- tests
- +-+- feature1
- | +--- positive.robot
- | +--- negative.robot
- +-+- feature2
- +--- positive.robot
- +--- negative.robot</code></pre></figure>
<p>To run test cases in specific test suiteuse <code class="highlighter-rouge">--suite</code> or <code class="highlighter-rouge">-s</code> option:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh"> <span class="c"># execute test cases from suites named "positive" in any file.
</span>
robot <span class="nt">--suite</span> positive <span class="nb">.</span>
<span class="c"># execute test cases from suite "feature1\positive" in any file.
</span>
robot <span class="nt">--suite</span> feature1.positive .</code></pre></figure>
<p>Also partial suite names and patterns can be used with <code class="highlighter-rouge">--suite</code> option, see <a href="#using-partial-names-and-filter-patterns">Using partial names and filter patterns</a>.</p>
<h1 id="execute-failed-tests">Execute failed tests</h1>
<p>There is also possibility to rerun all failed test cases and test suites.</p>
<h2 id="execute-only-failed-test-cases">Execute only failed test cases</h2>
<p>To rerun failed test cases use <code class="highlighter-rouge">--rerunfailed</code> or <code class="highlighter-rouge">-R</code> option:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh"> <span class="c"># execute test cases failed in previous run (saved in output.xml)
</span>
robot <span class="nt">--rerunfailed</span> output.xml <span class="nb">.</span> </code></pre></figure>
<h2 id="execute-failed-test-suites">Execute failed test suites</h2>
<p>To rerun test suites with failed test cases use <code class="highlighter-rouge">--rerunfailedsuites</code> or <code class="highlighter-rouge">-S</code> option:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh"> <span class="c"># execute test cases with failed test cases in previous run (saved in output.xml)
</span>
robot <span class="nt">--rerunfailedsuites</span> output.xml <span class="nb">.</span> </code></pre></figure>
<h1 id="using-partial-names-and-filter-patterns">Using partial names and filter patterns</h1>
<p>The presented <code class="highlighter-rouge">--test</code>, <code class="highlighter-rouge">--suite</code>, <code class="highlighter-rouge">--include</code> and <code class="highlighter-rouge">--exclude</code> options also support using partial names and filter patterns to match multiple names and tags:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh"> <span class="c"># execute test cases containing name "Example" in any file.
</span>
robot <span class="nt">--test</span> <span class="k">*</span>Example<span class="k">*</span> <span class="nb">.</span>
<span class="c"># execute test cases "Example One" and "Example Two" in any file.
</span>
robot <span class="nt">--test</span> <span class="s2">"Example [One|Two]"</span> <span class="nb">.</span>
<span class="c"># execute test cases with tags starting with "One" in any file.
</span>
robot <span class="nt">--include</span> One<span class="k">*</span> <span class="nb">.</span>
<span class="c"># execute test cases without tags ending with "Two" in any file.
</span>
robot <span class="nt">--exclude</span> <span class="k">*</span>Two <span class="nb">.</span>
<span class="c"># execute test cases from suites starting with "positive" in any file.
</span>
robot <span class="nt">--suite</span> positive<span class="k">*</span> .</code></pre></figure>
<p>For full list of supported filter patterns see User Guide section <a href="https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#simple-patterns" target="_blank">Simple Patterns</a></p>
<h1 id="combining-filters">Combining filters</h1>
<p>The presented <code class="highlighter-rouge">--test</code>, <code class="highlighter-rouge">--suite</code>, <code class="highlighter-rouge">--include</code> and <code class="highlighter-rouge">--exclude</code> options can be used also in combination:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh"> <span class="c"># execute test cases containing name "Example" and having tag "One" in any file.
</span>
robot <span class="nt">--include</span> One <span class="nt">--test</span> <span class="k">*</span>Example<span class="k">*</span> <span class="nb">.</span>
<span class="c"># execute test cases from suite "FeatureA" exluding tests with tag "Smoke" in any file.
</span>
robot <span class="nt">--suite</span> FeatureA <span class="nt">--exclude</span> Smoke <span class="nb">.</span>
<span class="c"># execute test cases with tag "Pending" from specific file.
</span>
robot <span class="nt">--exclude</span> Pending example.robot</code></pre></figure>jperalaYesterday a colleague learning test automation forgot the syntax for running Robot Framework test cases from command-line. Although full command-line syntax is described on the Robot Framework User Guide it is rather long and tedious document to dive into. This post aims to provide quick introduction for executing single, set or all Robot Framework test cases in a project. Basic syntax The basic syntax for executing Robot Framework tests from command line is: # basic syntax robot [options] robot_files # example with options robot -v URL:http://example.com example.robot For full list of command-line options see robot --help or robot -h option. Execute all test cases in folder(s) To run all robot tests in the folder (including subfolders) use . # execute all tests in all robot files in current folder and subfolders robot . Execute all test cases in single file To run all robot tests in single robot file: # execute all tests in single robot file in current folder robot example.robot # execute all tests in single robot file in subfolder robot path/to/example.robot Execute test cases by test name To run test cases with specific test name use --test or -t option: # execute test cases with name "Example" in any file. robot --test Example . # execute test cases with name "Example" in specific file. robot --test Example example.robot Also partial test names and patterns can be used with --test option, see Using partial names and filter patterns. Execute test cases by tags Test cases and suites annotated with tags (using [Tags] or Force Tags syntax) can be executed by selecting tags to be included or excluded. Example test suite: *** Settings *** Force Tags suite *** Test Cases *** Test One [Tags] one No operation Test Two [Tags] two No operation Also partial tag names and patterns can be used with --include and --exclude option, see Using partial names and filter patterns. Include test cases by tag To run test cases with specific tag name included use --include or -i option: # execute test cases with tag "one" in any file. robot --include one . # execute test cases with tags "one" and "two" in any file. robot --include oneANDtwo . # execute test cases with tag "one" or "two" in any file. robot --include oneORtwo . # execute test cases with tag "one" but without tag "two" in any file. robot --include oneNOTtwo . Exclude test cases by tag To run test cases with specific tag name excluded use --exclude or -e option: # execute test cases without tag "two" in any file. robot --exclude two . Execute test cases by suite name In Robot Framework test folders and .robot files are considered as test suites. Example suite structure: - +-+- tests - +-+- feature1 - | +--- positive.robot - | +--- negative.robot - +-+- feature2 - +--- positive.robot - +--- negative.robot To run test cases in specific test suiteuse --suite or -s option: # execute test cases from suites named "positive" in any file. robot --suite positive . # execute test cases from suite "feature1\positive" in any file. robot --suite feature1.positive . Also partial suite names and patterns can be used with --suite option, see Using partial names and filter patterns. Execute failed tests There is also possibility to rerun all failed test cases and test suites. Execute only failed test cases To rerun failed test cases use --rerunfailed or -R option: # execute test cases failed in previous run (saved in output.xml) robot --rerunfailed output.xml . Execute failed test suites To rerun test suites with failed test cases use --rerunfailedsuites or -S option: # execute test cases with failed test cases in previous run (saved in output.xml) robot --rerunfailedsuites output.xml . Using partial names and filter patterns The presented --test, --suite, --include and --exclude options also support using partial names and filter patterns to match multiple names and tags: # execute test cases containing name "Example" in any file. robot --test *Example* . # execute test cases "Example One" and "Example Two" in any file. robot --test "Example [One|Two]" . # execute test cases with tags starting with "One" in any file. robot --include One* . # execute test cases without tags ending with "Two" in any file. robot --exclude *Two . # execute test cases from suites starting with "positive" in any file. robot --suite positive* . For full list of supported filter patterns see User Guide section Simple Patterns Combining filters The presented --test, --suite, --include and --exclude options can be used also in combination: # execute test cases containing name "Example" and having tag "One" in any file. robot --include One --test *Example* . # execute test cases from suite "FeatureA" exluding tests with tag "Smoke" in any file. robot --suite FeatureA --exclude Smoke . # execute test cases with tag "Pending" from specific file. robot --exclude Pending example.robotReview Your Test Automation Code2018-09-11T00:34:00+03:002020-03-26T17:15:00+02:00https://jperala.fi/2018/09/11/why-to-review-test-automation-code<p>Doing test automation is about writing code. Test automation code can be easily treated as second-class citizen. As it’s not delivered to customer, development is often less formalized and may lack the scrutiny and quality practices otherwise applied in the organization. It’s also often first code many junior developers and testers start their journey into programming.</p>
<p>However, when it comes to creating valuable and maintainable automation, the automation code should be well implemented with maintainability in mind. Tests should be highly reliable as any false positives, false negatives or test flakiness will break the trusts towards the automation results.</p>
<p>One way to improve automation quality is to apply code review practices where automated tests are reviewed by your peers. With code review practices you can identify issues in automation that could affect test results and stability.</p>
<p>This post will present list of possible problem areas in automation code to look for in code reviews.</p>
<h1 id="things-to-look-out-in-code-review">Things to look out in code review</h1>
<h2 id="its-just-code">It’s just code</h2>
<p>As said, test automation is just code. Therefore common code review guidelines can be applied in the review process. Check out your organization review guidelines and practices if not familiar with doing reviews. For example, common things to look for include:</p>
<ul>
<li>Is the code understandable? Is there unnecessary complexity that could be simplified?</li>
<li>Are functions or classes too big, can those be broken down to smaller parts?</li>
<li>Does code units have too many responsibilities? Single responsibility principle?</li>
<li>Does the code follow DRY (Don’t Repeat Yourself) pattern? Is there duplication in the code?</li>
<li>Does the code follow common coding standards and conventions?</li>
</ul>
<h2 id="issues-specific-to-test-code">Issues specific to test code</h2>
<p>In addition of generic code review issues to look for, there are many problem areas and issues in the test logic that can be identified in code reviews. Following chapters describe few common categories of what to look out for.</p>
<p>The issues are demonstrated with code example using Selenium WebDriver and Java, but the presented issue categories are not specific to any particular language or automation framework per se.</p>
<h3 id="bad-assertions">Bad assertions</h3>
<p>One supprisingly common mistake seen in test scripts is forgotten assersions. Lack of assertion can make your test to produce false positive test results.</p>
<figure class="highlight"><pre><code class="language-java" data-lang="java"> <span class="kd">private</span> <span class="kt">void</span> <span class="nf">submitIsDisplayed</span><span class="o">()</span> <span class="o">{</span>
<span class="n">WebElement</span> <span class="n">button</span> <span class="o">=</span> <span class="n">driver</span><span class="o">.</span><span class="na">findElement</span><span class="o">(</span><span class="n">By</span><span class="o">.</span><span class="na">id</span><span class="o">(</span><span class="s">"submit"</span><span class="o">));</span>
<span class="c1">// check that element is displayed
</span>
<span class="n">button</span><span class="o">.</span><span class="na">isDisplayed</span><span class="o">();</span> <span class="c1">// MISSING ASSERTION
</span>
<span class="o">}</span></code></pre></figure>
<h3 id="retrys">Retrys</h3>
<p>Another issue commonly seen is trying to fix test stability issues by retrying test steps in case of exception. Usually this is a indication of brittle automation code and lack of understanding of the application internal behavior. Identify what is the application behaviour causing test step to fail and create needed wait or missing test logic to handle it reliably.</p>
<figure class="highlight"><pre><code class="language-java" data-lang="java"> <span class="kd">private</span> <span class="kt">void</span> <span class="nf">inputAmount</span><span class="o">(</span><span class="n">String</span> <span class="n">amount</span><span class="o">)</span> <span class="o">{</span>
<span class="k">try</span> <span class="o">{</span>
<span class="n">WebElement</span> <span class="n">field</span> <span class="o">=</span> <span class="n">driver</span><span class="o">.</span><span class="na">findElement</span><span class="o">(</span><span class="n">By</span><span class="o">.</span><span class="na">id</span><span class="o">(</span><span class="s">"amount"</span><span class="o">));</span>
<span class="n">field</span><span class="o">.</span><span class="na">sendKeys</span><span class="o">(</span><span class="n">amount</span><span class="o">);</span>
<span class="o">}</span> <span class="k">catch</span><span class="o">(</span><span class="n">StaleElementReferenceException</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
<span class="c1">// INPUT FAILED, RETRYING
</span>
<span class="n">WebElement</span> <span class="n">field</span> <span class="o">=</span> <span class="n">driver</span><span class="o">.</span><span class="na">findElement</span><span class="o">(</span><span class="n">By</span><span class="o">.</span><span class="na">id</span><span class="o">(</span><span class="s">"amount"</span><span class="o">));</span>
<span class="n">field</span><span class="o">.</span><span class="na">sendKeys</span><span class="o">(</span><span class="n">amount</span><span class="o">);</span>
<span class="o">}</span>
<span class="o">}</span></code></pre></figure>
<h3 id="sleeps-and-fixed-delays">Sleeps and fixed delays</h3>
<p>Very similar to retrys described above are fixed delays (sleeps) commonly seen to fix test stability issues. Identify what is the application event causing test step to fail and create appropriate wait for it. Don’t slow down tests with fixed delays.</p>
<figure class="highlight"><pre><code class="language-java" data-lang="java"> <span class="kd">private</span> <span class="kt">void</span> <span class="nf">acceptCookie</span><span class="o">()</span> <span class="kd">throws</span> <span class="n">InterruptedException</span> <span class="o">{</span>
<span class="c1">// FIXED DELAY
</span>
<span class="n">Thread</span><span class="o">.</span><span class="na">sleep</span><span class="o">(</span><span class="mi">5000</span><span class="o">);</span>
<span class="n">WebElement</span> <span class="n">cookieButton</span> <span class="o">=</span> <span class="n">driver</span><span class="o">.</span><span class="na">findElement</span><span class="o">(</span><span class="n">By</span><span class="o">.</span><span class="na">id</span><span class="o">(</span><span class="s">"accept"</span><span class="o">));</span>
<span class="n">cookieButton</span><span class="o">.</span><span class="na">click</span><span class="o">();</span>
<span class="o">}</span></code></pre></figure>
<h3 id="hard-coded-values-and-configurations">Hard-coded values and configurations</h3>
<p>One thing sure is that you never know where your tests will be run in the future. Therefore automation code should be easily configurable. What if you need to test in different environment? With different user? Or with different browser? Make your tests configurable from the get-go by avoiding fixed URLs, drivers, etc.</p>
<figure class="highlight"><pre><code class="language-java" data-lang="java"> <span class="kd">private</span> <span class="kt">void</span> <span class="nf">init</span><span class="o">()</span> <span class="o">{</span>
<span class="n">driver</span> <span class="o">=</span> <span class="k">new</span> <span class="n">FirefoxDriver</span><span class="o">();</span> <span class="c1">// FIXED BROWSER TYPE
</span>
<span class="n">driver</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="s">"https://192.168.1.5/app"</span><span class="o">);</span> <span class="c1">// FIXED URL
</span>
<span class="o">}</span></code></pre></figure>
<h3 id="wrong-level-of-abstraction">Wrong level of abstraction</h3>
<p>Commonly seen issue with BDD-based automation is wrong level of abstraction. In BDD the story files and scenarios should serve as examples describing your product behavior without detailed implementation mechanics - allowing common understanding (and means of communication) of product behaviour for different stakeholders.</p>
<p>Implementation specific scenarios often indicate also that BDD is only created for automation purposes rather than serving as common documentation for the whole development process. As such, having BDD layer in automation is merely just an extra boilerplate to be maintained with no real value.</p>
<figure class="highlight"><pre><code class="language-gherkin" data-lang="gherkin"><span class="kn">Scenario</span><span class="p">:</span> Application can be started
<span class="nf">Given </span>application is opened
<span class="nf">When </span>user clicks element <span class="s">"//button[@title='Start']"</span> <span class="c"># SPECIFICS TO UI IMPLEMENTATION
</span>
<span class="nf">Then </span>element <span class="s">"#app > div.started"</span> is visible <span class="c"># SPECIFICS TO UI IMPLEMENTATION</span></code></pre></figure>
<h3 id="lack-of-atomicity">Lack of atomicity</h3>
<p>Another thing to look out for is chained tests that can not be run without each other. Having chained tests make the understanding and debugging of tests more complicated and also causes failure propagation. Failure of single test can lead to failure of all the subsequent tests also.</p>
<figure class="highlight"><pre><code class="language-java" data-lang="java"><span class="nd">@FixMethodOrder</span><span class="o">(</span><span class="n">MethodSorters</span><span class="o">.</span><span class="na">NAME_ASCENDING</span><span class="o">)</span>
<span class="kd">public</span> <span class="kd">class</span> <span class="nc">ChainedTests</span> <span class="o">{</span>
<span class="c1">// ... init ....
</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">t1_canLoadPage</span><span class="o">()</span> <span class="o">{</span>
<span class="n">driver</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="n">getAppUrl</span><span class="o">());</span>
<span class="n">wait</span><span class="o">.</span><span class="na">until</span><span class="o">(</span><span class="n">ExpectedConditions</span><span class="o">.</span><span class="na">visibilityOfElementLocated</span><span class="o">(</span><span class="n">By</span><span class="o">.</span><span class="na">id</span><span class="o">(</span><span class="s">"password"</span><span class="o">)));</span>
<span class="o">}</span>
<span class="c1">// DEPENDENT ON T1_canLoadPage
</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">t2_canLogin</span><span class="o">()</span> <span class="o">{</span>
<span class="n">driver</span><span class="o">.</span><span class="na">findElement</span><span class="o">(</span><span class="n">By</span><span class="o">.</span><span class="na">id</span><span class="o">(</span><span class="s">"userid"</span><span class="o">)).</span><span class="na">sendKeys</span><span class="o">(</span><span class="n">userId</span><span class="o">);</span>
<span class="n">driver</span><span class="o">.</span><span class="na">findElement</span><span class="o">(</span><span class="n">By</span><span class="o">.</span><span class="na">id</span><span class="o">(</span><span class="s">"password"</span><span class="o">)).</span><span class="na">sendKeys</span><span class="o">(</span><span class="n">password</span><span class="o">);</span>
<span class="n">driver</span><span class="o">.</span><span class="na">findElement</span><span class="o">(</span><span class="n">By</span><span class="o">.</span><span class="na">id</span><span class="o">(</span><span class="s">"login"</span><span class="o">)).</span><span class="na">click</span><span class="o">();</span>
<span class="n">wait</span><span class="o">.</span><span class="na">until</span><span class="o">(</span><span class="n">ExpectedConditions</span><span class="o">.</span><span class="na">visibilityOfElementLocated</span><span class="o">(</span><span class="n">By</span><span class="o">.</span><span class="na">id</span><span class="o">(</span><span class="s">"order"</span><span class="o">)));</span>
<span class="o">}</span>
<span class="c1">// DEPENDENT ON T1_canLoadPage and T2_canLogin
</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">t3_canPlaceAnOrder</span><span class="o">()</span> <span class="o">{</span>
<span class="n">driver</span><span class="o">.</span><span class="na">findElement</span><span class="o">(</span><span class="n">By</span><span class="o">.</span><span class="na">id</span><span class="o">(</span><span class="s">"order"</span><span class="o">)).</span><span class="na">click</span><span class="o">();</span>
<span class="n">wait</span><span class="o">.</span><span class="na">until</span><span class="o">(</span><span class="n">ExpectedConditions</span><span class="o">.</span><span class="na">visibilityOfElementLocated</span><span class="o">(</span><span class="n">By</span><span class="o">.</span><span class="na">id</span><span class="o">(</span><span class="s">"confirmation"</span><span class="o">)));</span>
<span class="o">}</span>
<span class="o">}</span></code></pre></figure>
<h3 id="unreliable-selectors">Unreliable selectors</h3>
<p>UI-based tests usually use some sort of selectors to identify the elements to interact with. Using unreliable and vague selectors can make your tests brittle and cause test failures even in case of minor UI changes around your test scope. For example, using absolute xpath or css locator can break when extra tag is added in the UI as parent or sibling of an particular element.</p>
<p>Add unique locators in the product as needed to support testing. If this is not possible, identify stable ways to locate elements. Avoid using absolute css or xpath locators that are likely to break easily.</p>
<figure class="highlight"><pre><code class="language-java" data-lang="java"> <span class="kd">private</span> <span class="n">String</span> <span class="nf">getDisclaimer</span><span class="o">()</span> <span class="o">{</span>
<span class="c1">// ABSOLUTE XPATH LOCATOR THAT GETS EASILY BROKEN
</span>
<span class="n">By</span> <span class="n">disclaimer</span> <span class="o">=</span> <span class="n">By</span><span class="o">.</span><span class="na">xpath</span><span class="o">(</span><span class="s">"html/body/div[1]/section/div[1]/div/div/div/div[1]/div/div/div/div/div[3]/div[1]/div/h4"</span><span class="o">);</span>
<span class="k">return</span> <span class="n">driver</span><span class="o">.</span><span class="na">findElement</span><span class="o">(</span><span class="n">disclaimer</span><span class="o">).</span><span class="na">getText</span><span class="o">();</span>
<span class="o">}</span></code></pre></figure>
<h1 id="opportunity-for-learning">Opportunity for learning</h1>
<p>Code reviews should not be seen only as a mean to identify problems in code. Reading others code and getting feedback for your own is a great way to improve your automation skills. There’s always so much to learn from each others.</p>
<h1 id="means-to-an-end">Means to an End</h1>
<p>The presented list aims to identify issues that can cause your test to be brittle, false, or make them hard to maintain. However, it doesn’t aim to be ultimate truth about what is considered as bad practice. In some context using the presented approaches may be applicable or even only way to create workaround to get tests up and running. Sometimes it can be also reasoned for writing quick-n-dirty tests to verify something only once. Means to an end.</p>jperalaDoing test automation is about writing code. Test automation code can be easily treated as second-class citizen. As it’s not delivered to customer, development is often less formalized and may lack the scrutiny and quality practices otherwise applied in the organization. It’s also often first code many junior developers and testers start their journey into programming. However, when it comes to creating valuable and maintainable automation, the automation code should be well implemented with maintainability in mind. Tests should be highly reliable as any false positives, false negatives or test flakiness will break the trusts towards the automation results. One way to improve automation quality is to apply code review practices where automated tests are reviewed by your peers. With code review practices you can identify issues in automation that could affect test results and stability. This post will present list of possible problem areas in automation code to look for in code reviews. Things to look out in code review It’s just code As said, test automation is just code. Therefore common code review guidelines can be applied in the review process. Check out your organization review guidelines and practices if not familiar with doing reviews. For example, common things to look for include: Is the code understandable? Is there unnecessary complexity that could be simplified? Are functions or classes too big, can those be broken down to smaller parts? Does code units have too many responsibilities? Single responsibility principle? Does the code follow DRY (Don’t Repeat Yourself) pattern? Is there duplication in the code? Does the code follow common coding standards and conventions? Issues specific to test code In addition of generic code review issues to look for, there are many problem areas and issues in the test logic that can be identified in code reviews. Following chapters describe few common categories of what to look out for. The issues are demonstrated with code example using Selenium WebDriver and Java, but the presented issue categories are not specific to any particular language or automation framework per se. Bad assertions One supprisingly common mistake seen in test scripts is forgotten assersions. Lack of assertion can make your test to produce false positive test results. private void submitIsDisplayed() { WebElement button = driver.findElement(By.id("submit")); // check that element is displayed button.isDisplayed(); // MISSING ASSERTION } Retrys Another issue commonly seen is trying to fix test stability issues by retrying test steps in case of exception. Usually this is a indication of brittle automation code and lack of understanding of the application internal behavior. Identify what is the application behaviour causing test step to fail and create needed wait or missing test logic to handle it reliably. private void inputAmount(String amount) { try { WebElement field = driver.findElement(By.id("amount")); field.sendKeys(amount); } catch(StaleElementReferenceException e) { // INPUT FAILED, RETRYING WebElement field = driver.findElement(By.id("amount")); field.sendKeys(amount); } } Sleeps and fixed delays Very similar to retrys described above are fixed delays (sleeps) commonly seen to fix test stability issues. Identify what is the application event causing test step to fail and create appropriate wait for it. Don’t slow down tests with fixed delays. private void acceptCookie() throws InterruptedException { // FIXED DELAY Thread.sleep(5000); WebElement cookieButton = driver.findElement(By.id("accept")); cookieButton.click(); } Hard-coded values and configurations One thing sure is that you never know where your tests will be run in the future. Therefore automation code should be easily configurable. What if you need to test in different environment? With different user? Or with different browser? Make your tests configurable from the get-go by avoiding fixed URLs, drivers, etc. private void init() { driver = new FirefoxDriver(); // FIXED BROWSER TYPE driver.get("https://192.168.1.5/app"); // FIXED URL } Wrong level of abstraction Commonly seen issue with BDD-based automation is wrong level of abstraction. In BDD the story files and scenarios should serve as examples describing your product behavior without detailed implementation mechanics - allowing common understanding (and means of communication) of product behaviour for different stakeholders. Implementation specific scenarios often indicate also that BDD is only created for automation purposes rather than serving as common documentation for the whole development process. As such, having BDD layer in automation is merely just an extra boilerplate to be maintained with no real value. Scenario: Application can be started Given application is opened When user clicks element "//button[@title='Start']" # SPECIFICS TO UI IMPLEMENTATION Then element "#app > div.started" is visible # SPECIFICS TO UI IMPLEMENTATION Lack of atomicity Another thing to look out for is chained tests that can not be run without each other. Having chained tests make the understanding and debugging of tests more complicated and also causes failure propagation. Failure of single test can lead to failure of all the subsequent tests also. @FixMethodOrder(MethodSorters.NAME_ASCENDING) public class ChainedTests { // ... init .... @Test public void t1_canLoadPage() { driver.get(getAppUrl()); wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("password"))); } // DEPENDENT ON T1_canLoadPage @Test public void t2_canLogin() { driver.findElement(By.id("userid")).sendKeys(userId); driver.findElement(By.id("password")).sendKeys(password); driver.findElement(By.id("login")).click(); wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("order"))); } // DEPENDENT ON T1_canLoadPage and T2_canLogin @Test public void t3_canPlaceAnOrder() { driver.findElement(By.id("order")).click(); wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("confirmation"))); } } Unreliable selectors UI-based tests usually use some sort of selectors to identify the elements to interact with. Using unreliable and vague selectors can make your tests brittle and cause test failures even in case of minor UI changes around your test scope. For example, using absolute xpath or css locator can break when extra tag is added in the UI as parent or sibling of an particular element. Add unique locators in the product as needed to support testing. If this is not possible, identify stable ways to locate elements. Avoid using absolute css or xpath locators that are likely to break easily. private String getDisclaimer() { // ABSOLUTE XPATH LOCATOR THAT GETS EASILY BROKEN By disclaimer = By.xpath("html/body/div[1]/section/div[1]/div/div/div/div[1]/div/div/div/div/div[3]/div[1]/div/h4"); return driver.findElement(disclaimer).getText(); } Opportunity for learning Code reviews should not be seen only as a mean to identify problems in code. Reading others code and getting feedback for your own is a great way to improve your automation skills. There’s always so much to learn from each others. Means to an End The presented list aims to identify issues that can cause your test to be brittle, false, or make them hard to maintain. However, it doesn’t aim to be ultimate truth about what is considered as bad practice. In some context using the presented approaches may be applicable or even only way to create workaround to get tests up and running. Sometimes it can be also reasoned for writing quick-n-dirty tests to verify something only once. Means to an end.Ten Tips for Starting Test Automation2018-06-01T00:01:00+03:002020-03-26T17:14:00+02:00https://jperala.fi/2018/06/01/ten-tips-for-starting-test-automation<p><em>This is a re-post from an original two-post article posted in <a href="https://bitfactor.fi">Bitfactor Blog</a></em></p>
<hr />
<p><strong><em>Lost in test automation? Here is my selection of 10 tips for getting started and keeping your automation projects on the right track.</em></strong></p>
<p>In modern software development the need of test automation is evident. Latest development trends such as Continuous Delivery and DevOps aim towards ability for fast and iterative software releases, and automation of the development and release pipeline from developer workstation all the way to the production deployment. All this sets major challenges for testing and QA practices needed to ensure product quality. When release cycles are shortened from months to weeks and days, testing every release by hand is no longer a viable option. Even the basic regression testing of unchanged functionality can easily exhaust available testers, thus risking the release schedules and overall product quality.</p>
<p>Luckily, getting started with test automation is easy. There are numerous automation tools and frameworks and plenty of material available on the internet to get started. With these even testers with no previous experience on test automation development can get their first tests up and running relatively quickly.</p>
<p>However, what starts as a small and easy automation project tends to grow and become more complicated. As time goes by, new tests will be added, the need to support various product variants and versions becomes evident, test environments and tools change, people leave and join the project, and bunch of other unforeseen changes will happen. Without planning and clear focus automation projects can easily get bloated and side-tracked leading to exhaustive maintenance burden that will eventually reduce the benefits gained from the automation.</p>
<p>This post will give ten pointers for getting started and keeping your automation projects on the right track. The list is not complete, but highlights common culprits and challenges observed in many automation projects.</p>
<h1 id="automation-is-a-tool-for-testing-not-a-replacement">Automation is a tool for testing, not a replacement</h1>
<p>The common misunderstanding across the IT industry is that all testing should be automated. In reality there is no 100% test automation – or at least it will not be cost-effective.</p>
<p>Automated tests are merely checks that ensure software works as it was expected. As such, automation is an excellent tool for checking that nothing was broken when new product features were added and that new features work according to the specification. However, in practice it doesn’t make sense to check every imaginable detail of the product as it would eventually make your test suite unmaintainable. Automation also provides very limited or no support when it comes to identifying unknowns and unspecified product behaviour.</p>
<p>Gaining understanding whether the developed features really fulfil your customer needs in given context and constraints still requires plenty of human thinking, communication, collaboration and exploration. Create automation that enables testers to focus on the essentials instead of doing repetitive tasks that are well suited for automation.</p>
<p><strong><em>Exploratory testing paired up with smart automation strategy is a killer combination for bug hunting. Empower testers to focus on critical thinking and brainwork by letting automation take care of the repetitive tasks. Focus on automation that will bring most value to your team.</em></strong></p>
<h1 id="your-tests-are-a-product">Your tests are a product</h1>
<p>Many companies have well-defined processes and practices for developing quality software products, but may lack nearly all quality control when it comes down to developing and maintaining test assets. Test assets may be also developed and maintained by testers with limited software development and coding experience compared to full-time software developers.</p>
<p>Although tests are not typically delivered to your customers with product releases, the lifecycle of your tests can be as long as the lifecycle of your products. Tests written today may be used for years after product delivery to ensure future maintenance updates and releases.</p>
<p>To ensure your test assets are reliable, maintainable and easily extendable, good software development practices should be enforced. Also put focus on selecting right the automation tools and framework supporting your testing needs.</p>
<p>A few things to think about in test asset development:</p>
<ul>
<li>Version control, branching, tagging, linking tests to product versions.</li>
<li>Uniform and consistent coding convention.</li>
<li>Self-describing test code naming (classes, methods, variables, commits).</li>
<li>Configurability vs. hard-coded values and magic numbers.</li>
<li>Quality practices & learning: Code reviews, pair- and mob programming.</li>
</ul>
<p><strong><em>Writing test automation is software development – treat it as such. Focus on having good quality test assets that are easily maintainable and follow good development practices. Value of test suite is on providing information about the product. Having unreliable and ambiguous test suite creating misinformation can be worse than not having tests at all.</em></strong></p>
<h1 id="there-is-no-i-in-automation">There is no “I” in automation</h1>
<p>The value of any testing is on providing information about the product to stakeholders. That in mind, automation should never be only your personal quest.</p>
<p>Before starting out automation, get commitment from your team and organization. Define expected goals and scope for automation together, and include these in your team DoD (Definition of Done). Work together as a team, with support of both developers and testers, to achieve your goals.</p>
<p>Make sure the results of automated tests are always up-to-date and available to your team and other stakeholders so they have the latest information to support development decisions. Automate only tests that provide meaningful information; there’s no point in automating things that nobody cares about.</p>
<p><strong><em>Getting best value out of automation requires commitment from the whole team and organization. Work together. Create tests that bring value and meaningful information to your stakeholders.</em></strong></p>
<h1 id="solid-base">Solid base</h1>
<p>Most testers starting to dabble with automation have their first experience usually with UI (GUI) automation tools. In web context, it’s commonly Selenium WebDriver or some test framework using Selenium under the hood. It makes perfect sense: as a tester you obviously want to test the whole application end-to-end, with all nuts and bolts installed. And Selenium is a great tool for that with plenty of resources and examples to get started.</p>
<p>Alas, eventually you learn UI tests can be one of the most problematic tests to automate. First of all, they can be problematic regarding the project schedule. Testing the product via UI means you’ll need a working UI. However, that is often subject to continuous changes, fine-tuning and may be finalized in the very end of the project. Sometimes even the smallest UI changes can break your automation scripts, requiring excessive maintenance and refactoring work to keep up with the latest UI design. Secondly, implementation of UI based end-to-end tests with whole system in place have a tendency of being slower to execute and more brittle as there are more moving parts. Setting up test environment and test development is also usually more costly as there are more parts to configure and manage.</p>
<p>Often teams put too much value on testing everything end-to-end in realistic environment with production UI. Aim to start automation as early as possible in the scope the functionality is implemented. Shift left. Have plenty of precise tests with detailed granularity, keep feedback loop fast from testing to development, and aim to develop tests that are robust to unrelated changes. Work your way up having relevant tests in each level, all the way from single units, to APIs, and finally to the UI layer. Usually this approach is described as a test pyramid, an idea initially coined by Mike Cohn and few others, where you should have solid base of unit and integration tests checking most of the things to be covered in testing.</p>
<p>The test pyramid concept is also somewhat controversial and subject of numerous discussions in the community. My personal opinion is to use it as guideline, but not to take it too strictly as rule that can’t be broken. It’s not automatically wrong for example to have more end-to-end tests than integration tests automated. Also the cost and effort needed for solid unit test coverage can surprise in context where exhaustive use of mocking is required – test execution speed does not reflect the test implementation speed. Aim for shift left, don’t limit only to testing in one level, and adapt your way of working to find right balance of tests in your domain and context.</p>
<p><strong><em>Create solid base of unit and integration tests to catch most changes of the product, accompanied with smaller set of positive and negative end-to-end tests verifying overall functionality. Fill gaps with exploratory testing.</em></strong></p>
<h1 id="dont-fall-in-love-with-your-code">Don’t fall in love with your code</h1>
<p>It’s not uncommon to see test assets filled with obsolete information such unused classes and methods, commented-out code lines, outdated configuration files and obsolete test cases.</p>
<p>Having obsolete code and test data makes the project hard to learn, follow and work with. Enforce good coding practices keeping the code base clean of obsolete and deprecated code that can become source for misinformation and technical debt. Create daily habit to cleanup obsolete test assets from repository.</p>
<p><strong><em>Avoid technical debt and risk of misunderstanding. If test asset - whether it’s code, configuration, test, or anything else - becomes obsolete, thrash it. You have version control tools to keep track of the old versions in case you need to revert back to it.</em></strong></p>
<h1 id="as-weak-as-your-test-data">As weak as your test data</h1>
<p>To test deleting a bank account, you need to have a bank account created first. To test user login, you need a registered user information created. Practically all testing relies on the use of test data. How this data is created for testing purposes depends on the test scope and the context. In unit and integration tests, the test data may be generated using suitable mock interfaces and in-memory databases initialized at test setup, whereas a system-level test may rely on real database populated with SQL script, and so on.</p>
<p>Especially for end-to-end testing of complex systems it may not be always trivial to automate the creation of realistic test data, and you may be tempted to use manually created test data in your automated tests. Although using manually created test data may be an applicable solution in some cases, it has in general many limitations that may bite you back later. Manually created data may expire unless maintained periodically, you may not be able to edit or delete it if other tests rely on it, and running multiple tests utilizing same data can cause flakiness and unexpected behaviour. The worst case would be using shared test data, such as data used both by automation and exploratory testing, that may change its state uncontrollably.</p>
<p>Find ways to automate the creation of test data. Each test should rely on unique data that it may use, modify or delete on test execution. Ensure test data is always in a known state before test execution. Also remember to test the product with realistic data set sizes that are expected in production use.</p>
<p><strong><em>Success of automation depends on the quality of test data. Avoid use of manually configured, shared or uncontrollable test data. Find ways to mock, populate or virtualize data sources in your test environment to be suitable for your testing needs.</em></strong></p>
<h1 id="works-on-my-computer">Works on my computer</h1>
<p>I bet everyone has heard someone retort “Works on my computer”? Too much time is wasted on investigation and fixing issues related to differences in various test environment configurations causing the software behave unexpectedly.</p>
<p>Most often these problems are a consequence of manual setup and configuration of test environments leading to configuration nuances between different environments. Manual configuration is time consuming, prone to human errors, non-reproducible, and lacks rollback support to previous configurations. As a result, what works on your environment may not work on your colleagues’ - seemingly similar - environment and vice versa.</p>
<p>Another challenge are unstable test environments that may fail randomly during test setup and execution. Unstable environments create misinformation in terms of false test results, eat up credibility of the testing, and analysing these errors take valuable time away from actual testing work. In worst case these environmental issues can kill the benefits otherwise gained from automation.</p>
<p>Find ways to automate the setup and management of test environments. Use configuration management and virtualization tools (e.g. Docker, Puppet, etc.) to create infrastructure-as-code configuration that can be used throughout the development pipeline to setup reproducible development and testing environments for different phases and needs.</p>
<p><strong><em>Focus on creating a pipeline from developer workstation to production where environments work in similar manner. Avoid manual configurations and create automated scripts for creation, configuration and deployment of test environments with suitable tools.</em></strong></p>
<h1 id="timing-is-everything">Timing is everything</h1>
<p>The world is asynchronous. Whether your tests are expecting response from native GUI, web application (browser), API or proprietary interface, the expected events will most likely be happening in an asynchronous manner. If your tests are not prepared for this, they may become flaky and fail unexpectedly due unknown response times of the application under test.</p>
<p>Typical example of this is Selenium WebDriver based browser tests that can be extremely flaky unless appropriate waiting of correct web element state is used. In case the test does not wait an element to become present and visible before trying to interact with it, the test may fail unexpectedly. Too often these types of issues are resolved by adding fixed delay (sleep) before interaction. Although a common practise, it is a bad approach making the test slow to execute with no guarantee of stability unless excessively long delays are used. Preferred solution in this context would be to set up an explicit wait that waits until the element is ready for interaction.</p>
<p>Regardless of used test tools and context, it is important to understand the implications that timing handling has to your tests, and how to mitigate them correctly to avoid test flakiness. Identify asynchronous events in test interactions and apply solutions to handle them without fixed delays.</p>
<p><strong><em>In your tests, know what is expected to happen and wait for it. Don’t rely on fixed delays that will slow down test execution with no guarantee for stability.</em></strong></p>
<h1 id="simplify-isolate-the-unknown">Simplify, isolate the unknown</h1>
<p>When developing automated tests, keep them as simple as possible. Keep tests short and avoid adding too much content in a single test (suite). Create test suites that focus on a specific feature of your product.</p>
<p>Create tests that are atomic and well focused. Atomic tests are order independent and not dependent on the execution or results of other tests. Use suitable setup and teardown functions to keep the product in known state to achieve atomicity. Each test should have only one focus that it tests. Failure of test should explicitly identify the problematic part in the product without excessive result analysis.</p>
<p>Tests should also be resilient to changes outside the test scope. Avoid using product features outside your defined test scope that may be subject to unexpected changes. As an example, if you are testing an embedded application that is part of a larger web portal, create tests that focus only on that specific app. When interacting with the application under test, create selectors that are resilient to changes that may happen in the portal page outside current test scope.</p>
<p><strong><em>Keep your tests simple. Tests should tell directly what part of the product failed. Avoid vague shotgun tests without known scope. Create atomic tests that are resilient to changes happening outside the defined test scope to avoid test maintenance work and false negatives.</em></strong></p>
<h1 id="dont-repeat-yourself">Don’t repeat yourself</h1>
<p>When working with multiple projects, or even single project with multiple test cases, avoid writing duplicate test code. If the same test step or functionality is used in multiple test suites and projects, modularize the functionality as its own test library that can be imported and reused in other projects.</p>
<p>Creating reusable test modules and libraries brings multiple benefits. First, creating new projects will become easier as you will have a growing set of reusable functions available vs. always starting from the scratch. Second, it will ease the maintenance of the test assets when product functionality changes. When the functionality changes, the changes need to be done only in a single point in test code instead of refactoring all the related test projects and test cases.</p>
<p>As you start creating reusable test libraries for your product, don’t forget to document and communicate availability of those libraries within your teams and organization. Otherwise there is a risk of individual teams creating duplicate test libraries causing risk of confusion and extra maintenance effort.</p>
<p><strong><em>Reuse, reuse, reuse. Develop test modules and libraries to unify the way your tests behave. Avoid duplicate code and the need to fix same issue in multiple places.</em></strong></p>
<h1 id="final-thoughts">Final thoughts</h1>
<p>Getting started with test automation can feel overwhelming. If you are a tester starting out automation without earlier programming experience, the learning curve can be especially steep while learning both coding and automation basics in parallel. But don’t stop. Keep going, learn and remember that no one is a master in the beginning.</p>
<p>The great news is that there are great resources and communities to support your journey in automation. When stuck or unsure about the best way to continue, seek help. Check out available documentations, tutorials and resources on the Internet, read a book, or ask a question in testing communities such Ministry of Testing or Testers.io.</p>
<p>Also check out available training and commercial offerings near you. There are many great services and companies with automation expertise to support, coach and ensure successful adoption of test automation in your organization.</p>
<p><strong><em>Start, keep going and get connected. There are great communities, events, trainings & partners to support your automation challenge.</em></strong></p>jperalaThis is a re-post from an original two-post article posted in Bitfactor Blog Lost in test automation? Here is my selection of 10 tips for getting started and keeping your automation projects on the right track. In modern software development the need of test automation is evident. Latest development trends such as Continuous Delivery and DevOps aim towards ability for fast and iterative software releases, and automation of the development and release pipeline from developer workstation all the way to the production deployment. All this sets major challenges for testing and QA practices needed to ensure product quality. When release cycles are shortened from months to weeks and days, testing every release by hand is no longer a viable option. Even the basic regression testing of unchanged functionality can easily exhaust available testers, thus risking the release schedules and overall product quality. Luckily, getting started with test automation is easy. There are numerous automation tools and frameworks and plenty of material available on the internet to get started. With these even testers with no previous experience on test automation development can get their first tests up and running relatively quickly. However, what starts as a small and easy automation project tends to grow and become more complicated. As time goes by, new tests will be added, the need to support various product variants and versions becomes evident, test environments and tools change, people leave and join the project, and bunch of other unforeseen changes will happen. Without planning and clear focus automation projects can easily get bloated and side-tracked leading to exhaustive maintenance burden that will eventually reduce the benefits gained from the automation. This post will give ten pointers for getting started and keeping your automation projects on the right track. The list is not complete, but highlights common culprits and challenges observed in many automation projects. Automation is a tool for testing, not a replacement The common misunderstanding across the IT industry is that all testing should be automated. In reality there is no 100% test automation – or at least it will not be cost-effective. Automated tests are merely checks that ensure software works as it was expected. As such, automation is an excellent tool for checking that nothing was broken when new product features were added and that new features work according to the specification. However, in practice it doesn’t make sense to check every imaginable detail of the product as it would eventually make your test suite unmaintainable. Automation also provides very limited or no support when it comes to identifying unknowns and unspecified product behaviour. Gaining understanding whether the developed features really fulfil your customer needs in given context and constraints still requires plenty of human thinking, communication, collaboration and exploration. Create automation that enables testers to focus on the essentials instead of doing repetitive tasks that are well suited for automation. Exploratory testing paired up with smart automation strategy is a killer combination for bug hunting. Empower testers to focus on critical thinking and brainwork by letting automation take care of the repetitive tasks. Focus on automation that will bring most value to your team. Your tests are a product Many companies have well-defined processes and practices for developing quality software products, but may lack nearly all quality control when it comes down to developing and maintaining test assets. Test assets may be also developed and maintained by testers with limited software development and coding experience compared to full-time software developers. Although tests are not typically delivered to your customers with product releases, the lifecycle of your tests can be as long as the lifecycle of your products. Tests written today may be used for years after product delivery to ensure future maintenance updates and releases. To ensure your test assets are reliable, maintainable and easily extendable, good software development practices should be enforced. Also put focus on selecting right the automation tools and framework supporting your testing needs. A few things to think about in test asset development: Version control, branching, tagging, linking tests to product versions. Uniform and consistent coding convention. Self-describing test code naming (classes, methods, variables, commits). Configurability vs. hard-coded values and magic numbers. Quality practices & learning: Code reviews, pair- and mob programming. Writing test automation is software development – treat it as such. Focus on having good quality test assets that are easily maintainable and follow good development practices. Value of test suite is on providing information about the product. Having unreliable and ambiguous test suite creating misinformation can be worse than not having tests at all. There is no “I” in automation The value of any testing is on providing information about the product to stakeholders. That in mind, automation should never be only your personal quest. Before starting out automation, get commitment from your team and organization. Define expected goals and scope for automation together, and include these in your team DoD (Definition of Done). Work together as a team, with support of both developers and testers, to achieve your goals. Make sure the results of automated tests are always up-to-date and available to your team and other stakeholders so they have the latest information to support development decisions. Automate only tests that provide meaningful information; there’s no point in automating things that nobody cares about. Getting best value out of automation requires commitment from the whole team and organization. Work together. Create tests that bring value and meaningful information to your stakeholders. Solid base Most testers starting to dabble with automation have their first experience usually with UI (GUI) automation tools. In web context, it’s commonly Selenium WebDriver or some test framework using Selenium under the hood. It makes perfect sense: as a tester you obviously want to test the whole application end-to-end, with all nuts and bolts installed. And Selenium is a great tool for that with plenty of resources and examples to get started. Alas, eventually you learn UI tests can be one of the most problematic tests to automate. First of all, they can be problematic regarding the project schedule. Testing the product via UI means you’ll need a working UI. However, that is often subject to continuous changes, fine-tuning and may be finalized in the very end of the project. Sometimes even the smallest UI changes can break your automation scripts, requiring excessive maintenance and refactoring work to keep up with the latest UI design. Secondly, implementation of UI based end-to-end tests with whole system in place have a tendency of being slower to execute and more brittle as there are more moving parts. Setting up test environment and test development is also usually more costly as there are more parts to configure and manage. Often teams put too much value on testing everything end-to-end in realistic environment with production UI. Aim to start automation as early as possible in the scope the functionality is implemented. Shift left. Have plenty of precise tests with detailed granularity, keep feedback loop fast from testing to development, and aim to develop tests that are robust to unrelated changes. Work your way up having relevant tests in each level, all the way from single units, to APIs, and finally to the UI layer. Usually this approach is described as a test pyramid, an idea initially coined by Mike Cohn and few others, where you should have solid base of unit and integration tests checking most of the things to be covered in testing. The test pyramid concept is also somewhat controversial and subject of numerous discussions in the community. My personal opinion is to use it as guideline, but not to take it too strictly as rule that can’t be broken. It’s not automatically wrong for example to have more end-to-end tests than integration tests automated. Also the cost and effort needed for solid unit test coverage can surprise in context where exhaustive use of mocking is required – test execution speed does not reflect the test implementation speed. Aim for shift left, don’t limit only to testing in one level, and adapt your way of working to find right balance of tests in your domain and context. Create solid base of unit and integration tests to catch most changes of the product, accompanied with smaller set of positive and negative end-to-end tests verifying overall functionality. Fill gaps with exploratory testing. Don’t fall in love with your code It’s not uncommon to see test assets filled with obsolete information such unused classes and methods, commented-out code lines, outdated configuration files and obsolete test cases. Having obsolete code and test data makes the project hard to learn, follow and work with. Enforce good coding practices keeping the code base clean of obsolete and deprecated code that can become source for misinformation and technical debt. Create daily habit to cleanup obsolete test assets from repository. Avoid technical debt and risk of misunderstanding. If test asset - whether it’s code, configuration, test, or anything else - becomes obsolete, thrash it. You have version control tools to keep track of the old versions in case you need to revert back to it. As weak as your test data To test deleting a bank account, you need to have a bank account created first. To test user login, you need a registered user information created. Practically all testing relies on the use of test data. How this data is created for testing purposes depends on the test scope and the context. In unit and integration tests, the test data may be generated using suitable mock interfaces and in-memory databases initialized at test setup, whereas a system-level test may rely on real database populated with SQL script, and so on. Especially for end-to-end testing of complex systems it may not be always trivial to automate the creation of realistic test data, and you may be tempted to use manually created test data in your automated tests. Although using manually created test data may be an applicable solution in some cases, it has in general many limitations that may bite you back later. Manually created data may expire unless maintained periodically, you may not be able to edit or delete it if other tests rely on it, and running multiple tests utilizing same data can cause flakiness and unexpected behaviour. The worst case would be using shared test data, such as data used both by automation and exploratory testing, that may change its state uncontrollably. Find ways to automate the creation of test data. Each test should rely on unique data that it may use, modify or delete on test execution. Ensure test data is always in a known state before test execution. Also remember to test the product with realistic data set sizes that are expected in production use. Success of automation depends on the quality of test data. Avoid use of manually configured, shared or uncontrollable test data. Find ways to mock, populate or virtualize data sources in your test environment to be suitable for your testing needs. Works on my computer I bet everyone has heard someone retort “Works on my computer”? Too much time is wasted on investigation and fixing issues related to differences in various test environment configurations causing the software behave unexpectedly. Most often these problems are a consequence of manual setup and configuration of test environments leading to configuration nuances between different environments. Manual configuration is time consuming, prone to human errors, non-reproducible, and lacks rollback support to previous configurations. As a result, what works on your environment may not work on your colleagues’ - seemingly similar - environment and vice versa. Another challenge are unstable test environments that may fail randomly during test setup and execution. Unstable environments create misinformation in terms of false test results, eat up credibility of the testing, and analysing these errors take valuable time away from actual testing work. In worst case these environmental issues can kill the benefits otherwise gained from automation. Find ways to automate the setup and management of test environments. Use configuration management and virtualization tools (e.g. Docker, Puppet, etc.) to create infrastructure-as-code configuration that can be used throughout the development pipeline to setup reproducible development and testing environments for different phases and needs. Focus on creating a pipeline from developer workstation to production where environments work in similar manner. Avoid manual configurations and create automated scripts for creation, configuration and deployment of test environments with suitable tools. Timing is everything The world is asynchronous. Whether your tests are expecting response from native GUI, web application (browser), API or proprietary interface, the expected events will most likely be happening in an asynchronous manner. If your tests are not prepared for this, they may become flaky and fail unexpectedly due unknown response times of the application under test. Typical example of this is Selenium WebDriver based browser tests that can be extremely flaky unless appropriate waiting of correct web element state is used. In case the test does not wait an element to become present and visible before trying to interact with it, the test may fail unexpectedly. Too often these types of issues are resolved by adding fixed delay (sleep) before interaction. Although a common practise, it is a bad approach making the test slow to execute with no guarantee of stability unless excessively long delays are used. Preferred solution in this context would be to set up an explicit wait that waits until the element is ready for interaction. Regardless of used test tools and context, it is important to understand the implications that timing handling has to your tests, and how to mitigate them correctly to avoid test flakiness. Identify asynchronous events in test interactions and apply solutions to handle them without fixed delays. In your tests, know what is expected to happen and wait for it. Don’t rely on fixed delays that will slow down test execution with no guarantee for stability. Simplify, isolate the unknown When developing automated tests, keep them as simple as possible. Keep tests short and avoid adding too much content in a single test (suite). Create test suites that focus on a specific feature of your product. Create tests that are atomic and well focused. Atomic tests are order independent and not dependent on the execution or results of other tests. Use suitable setup and teardown functions to keep the product in known state to achieve atomicity. Each test should have only one focus that it tests. Failure of test should explicitly identify the problematic part in the product without excessive result analysis. Tests should also be resilient to changes outside the test scope. Avoid using product features outside your defined test scope that may be subject to unexpected changes. As an example, if you are testing an embedded application that is part of a larger web portal, create tests that focus only on that specific app. When interacting with the application under test, create selectors that are resilient to changes that may happen in the portal page outside current test scope. Keep your tests simple. Tests should tell directly what part of the product failed. Avoid vague shotgun tests without known scope. Create atomic tests that are resilient to changes happening outside the defined test scope to avoid test maintenance work and false negatives. Don’t repeat yourself When working with multiple projects, or even single project with multiple test cases, avoid writing duplicate test code. If the same test step or functionality is used in multiple test suites and projects, modularize the functionality as its own test library that can be imported and reused in other projects. Creating reusable test modules and libraries brings multiple benefits. First, creating new projects will become easier as you will have a growing set of reusable functions available vs. always starting from the scratch. Second, it will ease the maintenance of the test assets when product functionality changes. When the functionality changes, the changes need to be done only in a single point in test code instead of refactoring all the related test projects and test cases. As you start creating reusable test libraries for your product, don’t forget to document and communicate availability of those libraries within your teams and organization. Otherwise there is a risk of individual teams creating duplicate test libraries causing risk of confusion and extra maintenance effort. Reuse, reuse, reuse. Develop test modules and libraries to unify the way your tests behave. Avoid duplicate code and the need to fix same issue in multiple places. Final thoughts Getting started with test automation can feel overwhelming. If you are a tester starting out automation without earlier programming experience, the learning curve can be especially steep while learning both coding and automation basics in parallel. But don’t stop. Keep going, learn and remember that no one is a master in the beginning. The great news is that there are great resources and communities to support your journey in automation. When stuck or unsure about the best way to continue, seek help. Check out available documentations, tutorials and resources on the Internet, read a book, or ask a question in testing communities such Ministry of Testing or Testers.io. Also check out available training and commercial offerings near you. There are many great services and companies with automation expertise to support, coach and ensure successful adoption of test automation in your organization. Start, keep going and get connected. There are great communities, events, trainings & partners to support your automation challenge.Getting started on Web Development2018-01-26T22:43:00+02:002020-03-26T17:13:00+02:00https://jperala.fi/2018/01/26/getting-started-on-web-developent<p>It kind of shocked me to realize that I’ve never really developed frontend web applications - There just haven’t been any need for it. For the most of my career I’ve worked on job roles related to test automation. That includes plenty of programming and development on daily basis, but all the development I do is for testing. Meaning things like writing automated tests using frameworks such Selenium, JUnit and Robot Framework; creating automation scripts for setting up test environments; developing test tools; and so on. I’ve also worked purely on development roles, but always related to embedded or backend development.</p>
<p>Inspired by this “revelation” I decided to start an side-project to get more familiar on developing frontend applications with modern tools. It would give me also a great opportunity to try out cloud platforms and their benefits from app development perspective - for now I’ve used cloud platforms only for testing related tasks.</p>
<p>Here’s story of what I did and thoughs throughout the learning path. The post does not aim to be a step-by-step tutorial, but rather give an overview of what to look into when starting frontend web development.</p>
<h2 id="the-goal">The Goal</h2>
<p>I wanted to develop some application that could potentially be somewhat usefull for others also. Maybe something that uses Open Data.</p>
<p>Few years ago a massive undeground parking station (Kivisydän) was opened in Oulu and I’ve always been kind of curious does it ever get fully crowded. As it happens to be, the City of Oulu exposes the realtime utilization of parking stations as <a href="https://data.ouka.fi">Open Data</a> API, so I decided to write an application that would store this information and show the parking stations current utilization and statistics in a web application.</p>
<p>As I wanted to store the utilization information for statistics, It was clear that I would need to implement the frontend application that shows this information, and also backend functionality that fetches data from public API, stores it database and provides API for the frontend application to query stored data.</p>
<h2 id="what-tools-for-development">What tools for development?</h2>
<p>Getting started on web development can be a bit painful. Even though I have plenty background with other programming and as a tester I’ve worked a lot with web apps to know solid basics of HTML markups and CSS styling, it wasn’t fully clear what tools and libraries I need to get started. As a new to frontend development it kind of felt that there is countless number of .js libraries and tools for everything and I felt bit unsure to know what tools would I need and what tools overlap with others.</p>
<p>One great resource describing the tools and frameworks in web development that I found was Matt Raible’s Devoxx talk <a href="https://www.youtube.com/watch?v=AjzyV8BraIs">“Front Ends for Back End Developers”</a>. That is definitely worth watching.</p>
<p>After a bit of googling and familiarization, it started to look that the things I would probably need to choose or atleast be aware of are:</p>
<ul>
<li>Programming language</li>
<li>Application Framework</li>
<li>Dependency management tool</li>
<li>Build tool</li>
<li>Test tool</li>
<li>Bundler</li>
<li>Transpiler</li>
</ul>
<h3 id="programming-language">Programming language</h3>
<p>With programming languages you have few options, whether to use plain JavaScript or some alternate language like <a href="https://www.typescriptlang.org/">TypeScript</a> that compiles into JavaScript. With Java background I am more used to languages with static types so TypeScript sounded like an great way to go. However, to start with the basics I decided to use plain JavaScript, after that it would be straighforward to use other JavaScript-compilable languages also.</p>
<p>What comes to JavaScript, it’s also good to be aware of alternate versions supported. The ES5 is the “old-school” JavaScript supported by pretty much every browser. The ES6 (ES2015) changed the syntax quite a bit and introduced entities like classes to JavaScript. Also ES6 is nowadays well covered, but features introduced in ES2016 and beyond may not work on all browsers and environments (see <a href="http://kangax.github.io/compat-table/">compatibility table</a>). The good thing is, that even the newer features can be used with transpilers (e.g. Babel) that rewrite the code to be compatible with older JavaScript versions.</p>
<p>Besides JavaScript and TypeScript, there is also other languages to look into such <a href="http://elm-lang.org/">ELM</a>, <a href="http://www.purescript.org/">PureScript</a>, <a href="https://clojurescript.org/">ClojureScript</a>, <a href="http://coffeescript.org/">CoffeeScript</a>, and so on. Be open-minded and try what fits you the best.</p>
<h3 id="application-framework">Application Framework</h3>
<p>You can develop applications with plain (vanilla) JavaScript, but most projects use some kind of web framework for project structure. The top-3 currently seems to be <a href="https://reactjs.org/">React</a>, <a href="https://angular.io/">Angular</a> and <a href="https://vuejs.org/">Vue.js</a>. For any of these you are likely to find good community support with examples, forums, etc. There is also endless amount of other frameworks, <a href="https://www.meteor.com/">Meteor</a>, <a href="https://www.emberjs.com/">Ember.js</a> and <a href="https://cycle.js.org/">Cycle.js</a> to mention few.</p>
<p>For my project I chose React due it’s wide popularity and having colleagues with experience on it.</p>
<h3 id="dependency--build-tools">Dependency & Build Tools</h3>
<p>When developing your application you need to somehow manage and download the needed dependencies and tools. For this you most likely end-up using <a href="https://www.npmjs.com/">npm</a> or <a href="https://yarnpkg.com/en/">yarn</a>. Npm is the good ‘ol package manager that most tutorials seems to be using. Yarn is newer tool aiming to solve some problems and performance issues npm has.</p>
<p>Both npm and yarn can be also as build tools by running project-specific scripts with run command. In addition there is dedicated task runner tools like <a href="https://gulpjs.com/">Gulp</a> and <a href="https://gruntjs.com/">Grunt</a> to carry out more complex build tasks.</p>
<p>In my project I ended using npm but will move to yarn for future versions.</p>
<h3 id="test-tools">Test tools</h3>
<p>For testing your application there is an variety of JavaScript based test tools available that integrate into your build configuration. Some potential testing tools to checkout:</p>
<ul>
<li><a href="https://facebook.github.io/jest/">Jest</a> - default test with React’s <a href="https://github.com/facebook/create-react-app">create-react-app</a>. Works with any JavaScript code.</li>
<li><a href="http://www.protractortest.org/">Protractor</a> - End-to-end testing framework for Angular applications.</li>
<li><a href="https://jasmine.github.io/">Jasmine</a> - Behaviour driven development framework for testing JavaScript. Works with any JavaScript code.</li>
<li><a href="https://www.cypress.io/">Cypress.io</a> - Complete end-to-end testing for anything running in browser.</li>
<li><a href="http://nightwatchjs.org/">Nightwatch.io</a> - Browser automated testing done easy.</li>
</ul>
<p>For testing there is no reason to limit only on JavaScript based test tools, but use any tool you find suitable. For example <a href="https://www.getpostman.com/">Postman</a> is a great tool testing APIs, no matter what the implementation language is.</p>
<p>I used Jest that came preconfigured with React’s create-react-app and also Postman for API testing.</p>
<h3 id="bundler">Bundler</h3>
<p>When you want to run your application in a browser, you need to deliver all the dependent code modules to be able to execute your code. In very small projects with minimal dependencies you could just include your scripts as is, but for anything bigger you want to bundle up your code (and resources) to minimize the http requests and size of file downloads. For this there is tools such <a href="https://webpack.github.io/">Webpack</a> and <a href="http://browserify.org/">Browserify</a>.</p>
<p>I used Webpack that came preconfigured with React’s create-react-app.</p>
<h3 id="transpiler">Transpiler</h3>
<p>When using the latest features not yet supported by all targeted browsers, you may need to transpile the code to be compatible with older JavaScript versions.</p>
<p><a href="https://babeljs.io/">Babel</a> is the de-facto tool to do it.</p>
<h2 id="what-tools-for-cloud-deployment">What Tools for Cloud Deployment?</h2>
<p>Once it was clear what tools to use for developing the frontend app, I needed to think about deployment. I didn’t want to setup my own server, or virtual server that would require maintenance and create continuous costs for hosting the application.</p>
<p>This in mind I wanted to use cloud platform such <a href="https://aws.amazon.com/">Amazon AWS</a>, <a href="https://azure.microsoft.com/">Microsoft Azure</a> or <a href="https://cloud.google.com/">Google Cloud Platform</a>. For the backend application I also wanted to use serverless infrastructures that would enable deployment of the API as cloud functions billed by usage.</p>
<p>All big three cloud platforms provide quite comparable features and pricing, but for now I chose to use Amazon AWS. To deploy and host the application I used the AWS services described below.</p>
<h3 id="hosting-static-content">Hosting Static Content</h3>
<p>The created React application is client-side application that access dynamic content via API. For the static files of the React application I used <a href="https://aws.amazon.com/s3/">AWS S3</a> bucket to store files. You can either use S3 directly to host your website, or AWS Cloudfront CDN to deliver the content from S3 bucket.</p>
<h3 id="domain--dns--https">Domain & DNS & HTTPS</h3>
<p>I wanted to deploy my application in own domain oulunparkit.com. In AWS you can use <a href="https://aws.amazon.com/route53/">AWS Route 53</a> service to buy domain and setup hosted zone for DNS routing. To add support for SSL/TLS security for HTTPS, there is <a href="https://aws.amazon.com/certificate-manager/">AWS Certificate Manager</a> to create custom certificate.</p>
<h3 id="content-distribution">Content Distribution</h3>
<p>To speed up delivering content from S3 storage you can use <a href="https://aws.amazon.com/cloudfront/">AWS Cloudfront</a> CDN network to distribute and cache content from multiple edge locations. In case you are using custom domain and want to enable HTTPS access to content served from S3 storage, you need to create CloudFront distribution to handle custom certificate.</p>
<h3 id="api-functions--deployment">API Functions & Deployment</h3>
<p>To create API for data queries you need a place to deploy your API functions. <a href="https://aws.amazon.com/lambda/">AWS Lambda</a> provides serverless function service to execute functions written in Node.js, Java, .NET, Python or Go. To expose the Lambda function as external API you can use the <a href="https://aws.amazon.com/api-gateway/">AWS API Gateway</a>.</p>
<h3 id="database">Database</h3>
<p>For the API data storage we need a database. In AWS you have options such <a href="https://aws.amazon.com/dynamodb/">AWS DynamoDB</a>, <a href="https://aws.amazon.com/rds/">AWS RDS</a> and <a href="https://aws.amazon.com/rds/aurora/">AWS Aurora</a>. Depending whether you need transactional or no-sql database and type of features and pricing any of these would be an option.</p>
<p>In my application I decided to use AWS DynamoDB as database.</p>
<h3 id="aws-cli">AWS CLI</h3>
<p>Configuring and deploying AWS services can be done via AWS Management Console web app. However, for development using it becomes easily cumbersome, so taking the AWS CLI into use is definitely worth looking into. It enables you to easily create, modify and deploy AWS services and instances via command-line interface.</p>
<h2 id="developing-the-application">Developing the Application</h2>
<p>After it was clear what development tools and cloud services to use it was time to sink into tutorials to recap JavaScript language features and dive into React. Starting development with React was surprisingly straigthforward with the <a href="https://reactjs.org/tutorial/tutorial.html">Tic-Tac-Toe tutorial</a> and using the the create-react-app tool that creates a ready project skeleton with Babel, Webpack, Jest and other tools preconfigured.</p>
<p>The great thing with React is also a vast amount of ready made components to use in your application. For example, to create graphs I used <a href="http://recharts.org/">ReCharts</a>, for drawing Google Map there is <a href="https://github.com/istarkov/google-map-react">google-react-map</a> and for tabs there was <a href="https://github.com/reactjs/react-tabs">react-tabs</a>. I also added localization support (FI and EN) using the <a href="https://github.com/stefalda/react-localization">react-localization</a>.</p>
<p>One thing to think about is the styling of the components. You can write your own components with custom CSS, but there is also many responsive UI frameworks such <a href="http://www.material-ui.com/">Material-UI</a>, <a href="http://react.semantic-ui.com/">Semantic-UI</a> and <a href="https://react-bootstrap.github.io/">React-Bootstrap</a>.</p>
<p>Developing the API functions for AWS Lambda was also very straighforward. AWS provides ready-made blueprints configuring the lambda for use as microservice backed with DynamoDB.</p>
<h2 id="the-result-oulunparkitcom">The Result: oulunparkit.com</h2>
<p>Well, it’s still lacks some features and UI finishing, but the first versions of the app is released in <a href="https://oulunparkit.com/">oulunparkit.com</a>.</p>
<p>In the application you can see the current utilization of parking stations either in map or list view. By selecting an parking station you can dive into station details and see the utilization statistics for the last 1, 7 or 30 days as a graph.</p>
<p>The plan is to update the app to allow users to input free-range statistics period to be shown. Currently the app uses tiny responsive CSS framework called <a href="https://ajusa.github.io/lit/">lit</a>, but the plan is to dabble with Material-UI and alternative UI frameworks to improve the UI look & feel. I also intend to rewrite the app behaviour to play around with React-redux and React-router modules.</p>
<p>The sources are available in my GitHub repository <a href="https://github.com/juperala/parking-oulu">parking-oulu</a>.</p>
<h2 id="wrap-up">Wrap-up</h2>
<p>In overall developing and deploying a full application was a very learning experience. There wasn’t any major obstacles on the development, only new tools to learn about. For any problem there was great amount of resources available with a bit of googling.</p>
<p>Definitely the biggest challenge was to start and get high-level understanding of what tools to be used for what task. After you known what you need to accomplish, for any of the used tools used there was plenty of materials available to dive into details and find the solution. And when unsure what tool does or how it works, just go through the tutorials.</p>
<p>It was also nice to find out that working in test automation development doesn’t differ so much from application development. Sure, the languages, tools and libraries used were mostly new and it takes time to master them deeply. But after all it’s just about understanding what tools/libraries to use for what purpose and how to use them together.</p>jperalaIt kind of shocked me to realize that I’ve never really developed frontend web applications - There just haven’t been any need for it. For the most of my career I’ve worked on job roles related to test automation. That includes plenty of programming and development on daily basis, but all the development I do is for testing. Meaning things like writing automated tests using frameworks such Selenium, JUnit and Robot Framework; creating automation scripts for setting up test environments; developing test tools; and so on. I’ve also worked purely on development roles, but always related to embedded or backend development. Inspired by this “revelation” I decided to start an side-project to get more familiar on developing frontend applications with modern tools. It would give me also a great opportunity to try out cloud platforms and their benefits from app development perspective - for now I’ve used cloud platforms only for testing related tasks. Here’s story of what I did and thoughs throughout the learning path. The post does not aim to be a step-by-step tutorial, but rather give an overview of what to look into when starting frontend web development. The Goal I wanted to develop some application that could potentially be somewhat usefull for others also. Maybe something that uses Open Data. Few years ago a massive undeground parking station (Kivisydän) was opened in Oulu and I’ve always been kind of curious does it ever get fully crowded. As it happens to be, the City of Oulu exposes the realtime utilization of parking stations as Open Data API, so I decided to write an application that would store this information and show the parking stations current utilization and statistics in a web application. As I wanted to store the utilization information for statistics, It was clear that I would need to implement the frontend application that shows this information, and also backend functionality that fetches data from public API, stores it database and provides API for the frontend application to query stored data. What tools for development? Getting started on web development can be a bit painful. Even though I have plenty background with other programming and as a tester I’ve worked a lot with web apps to know solid basics of HTML markups and CSS styling, it wasn’t fully clear what tools and libraries I need to get started. As a new to frontend development it kind of felt that there is countless number of .js libraries and tools for everything and I felt bit unsure to know what tools would I need and what tools overlap with others. One great resource describing the tools and frameworks in web development that I found was Matt Raible’s Devoxx talk “Front Ends for Back End Developers”. That is definitely worth watching. After a bit of googling and familiarization, it started to look that the things I would probably need to choose or atleast be aware of are: Programming language Application Framework Dependency management tool Build tool Test tool Bundler Transpiler Programming language With programming languages you have few options, whether to use plain JavaScript or some alternate language like TypeScript that compiles into JavaScript. With Java background I am more used to languages with static types so TypeScript sounded like an great way to go. However, to start with the basics I decided to use plain JavaScript, after that it would be straighforward to use other JavaScript-compilable languages also. What comes to JavaScript, it’s also good to be aware of alternate versions supported. The ES5 is the “old-school” JavaScript supported by pretty much every browser. The ES6 (ES2015) changed the syntax quite a bit and introduced entities like classes to JavaScript. Also ES6 is nowadays well covered, but features introduced in ES2016 and beyond may not work on all browsers and environments (see compatibility table). The good thing is, that even the newer features can be used with transpilers (e.g. Babel) that rewrite the code to be compatible with older JavaScript versions. Besides JavaScript and TypeScript, there is also other languages to look into such ELM, PureScript, ClojureScript, CoffeeScript, and so on. Be open-minded and try what fits you the best. Application Framework You can develop applications with plain (vanilla) JavaScript, but most projects use some kind of web framework for project structure. The top-3 currently seems to be React, Angular and Vue.js. For any of these you are likely to find good community support with examples, forums, etc. There is also endless amount of other frameworks, Meteor, Ember.js and Cycle.js to mention few. For my project I chose React due it’s wide popularity and having colleagues with experience on it. Dependency & Build Tools When developing your application you need to somehow manage and download the needed dependencies and tools. For this you most likely end-up using npm or yarn. Npm is the good ‘ol package manager that most tutorials seems to be using. Yarn is newer tool aiming to solve some problems and performance issues npm has. Both npm and yarn can be also as build tools by running project-specific scripts with run command. In addition there is dedicated task runner tools like Gulp and Grunt to carry out more complex build tasks. In my project I ended using npm but will move to yarn for future versions. Test tools For testing your application there is an variety of JavaScript based test tools available that integrate into your build configuration. Some potential testing tools to checkout: Jest - default test with React’s create-react-app. Works with any JavaScript code. Protractor - End-to-end testing framework for Angular applications. Jasmine - Behaviour driven development framework for testing JavaScript. Works with any JavaScript code. Cypress.io - Complete end-to-end testing for anything running in browser. Nightwatch.io - Browser automated testing done easy. For testing there is no reason to limit only on JavaScript based test tools, but use any tool you find suitable. For example Postman is a great tool testing APIs, no matter what the implementation language is. I used Jest that came preconfigured with React’s create-react-app and also Postman for API testing. Bundler When you want to run your application in a browser, you need to deliver all the dependent code modules to be able to execute your code. In very small projects with minimal dependencies you could just include your scripts as is, but for anything bigger you want to bundle up your code (and resources) to minimize the http requests and size of file downloads. For this there is tools such Webpack and Browserify. I used Webpack that came preconfigured with React’s create-react-app. Transpiler When using the latest features not yet supported by all targeted browsers, you may need to transpile the code to be compatible with older JavaScript versions. Babel is the de-facto tool to do it. What Tools for Cloud Deployment? Once it was clear what tools to use for developing the frontend app, I needed to think about deployment. I didn’t want to setup my own server, or virtual server that would require maintenance and create continuous costs for hosting the application. This in mind I wanted to use cloud platform such Amazon AWS, Microsoft Azure or Google Cloud Platform. For the backend application I also wanted to use serverless infrastructures that would enable deployment of the API as cloud functions billed by usage. All big three cloud platforms provide quite comparable features and pricing, but for now I chose to use Amazon AWS. To deploy and host the application I used the AWS services described below. Hosting Static Content The created React application is client-side application that access dynamic content via API. For the static files of the React application I used AWS S3 bucket to store files. You can either use S3 directly to host your website, or AWS Cloudfront CDN to deliver the content from S3 bucket. Domain & DNS & HTTPS I wanted to deploy my application in own domain oulunparkit.com. In AWS you can use AWS Route 53 service to buy domain and setup hosted zone for DNS routing. To add support for SSL/TLS security for HTTPS, there is AWS Certificate Manager to create custom certificate. Content Distribution To speed up delivering content from S3 storage you can use AWS Cloudfront CDN network to distribute and cache content from multiple edge locations. In case you are using custom domain and want to enable HTTPS access to content served from S3 storage, you need to create CloudFront distribution to handle custom certificate. API Functions & Deployment To create API for data queries you need a place to deploy your API functions. AWS Lambda provides serverless function service to execute functions written in Node.js, Java, .NET, Python or Go. To expose the Lambda function as external API you can use the AWS API Gateway. Database For the API data storage we need a database. In AWS you have options such AWS DynamoDB, AWS RDS and AWS Aurora. Depending whether you need transactional or no-sql database and type of features and pricing any of these would be an option. In my application I decided to use AWS DynamoDB as database. AWS CLI Configuring and deploying AWS services can be done via AWS Management Console web app. However, for development using it becomes easily cumbersome, so taking the AWS CLI into use is definitely worth looking into. It enables you to easily create, modify and deploy AWS services and instances via command-line interface. Developing the Application After it was clear what development tools and cloud services to use it was time to sink into tutorials to recap JavaScript language features and dive into React. Starting development with React was surprisingly straigthforward with the Tic-Tac-Toe tutorial and using the the create-react-app tool that creates a ready project skeleton with Babel, Webpack, Jest and other tools preconfigured. The great thing with React is also a vast amount of ready made components to use in your application. For example, to create graphs I used ReCharts, for drawing Google Map there is google-react-map and for tabs there was react-tabs. I also added localization support (FI and EN) using the react-localization. One thing to think about is the styling of the components. You can write your own components with custom CSS, but there is also many responsive UI frameworks such Material-UI, Semantic-UI and React-Bootstrap. Developing the API functions for AWS Lambda was also very straighforward. AWS provides ready-made blueprints configuring the lambda for use as microservice backed with DynamoDB. The Result: oulunparkit.com Well, it’s still lacks some features and UI finishing, but the first versions of the app is released in oulunparkit.com. In the application you can see the current utilization of parking stations either in map or list view. By selecting an parking station you can dive into station details and see the utilization statistics for the last 1, 7 or 30 days as a graph. The plan is to update the app to allow users to input free-range statistics period to be shown. Currently the app uses tiny responsive CSS framework called lit, but the plan is to dabble with Material-UI and alternative UI frameworks to improve the UI look & feel. I also intend to rewrite the app behaviour to play around with React-redux and React-router modules. The sources are available in my GitHub repository parking-oulu. Wrap-up In overall developing and deploying a full application was a very learning experience. There wasn’t any major obstacles on the development, only new tools to learn about. For any problem there was great amount of resources available with a bit of googling. Definitely the biggest challenge was to start and get high-level understanding of what tools to be used for what task. After you known what you need to accomplish, for any of the used tools used there was plenty of materials available to dive into details and find the solution. And when unsure what tool does or how it works, just go through the tutorials. It was also nice to find out that working in test automation development doesn’t differ so much from application development. Sure, the languages, tools and libraries used were mostly new and it takes time to master them deeply. But after all it’s just about understanding what tools/libraries to use for what purpose and how to use them together.Summary of #30DaysOfTesting2016-07-31T17:34:00+03:002020-03-26T17:12:00+02:00https://jperala.fi/2016/07/31/30-days-of-testing<p><em>Ministry of Testing</em> started a <a href="http://www.ministryoftesting.com/2016/06/30-day-testing-challenge/">#30DaysOfTesting</a> challenge for the July. I decided to join the challenge as it provided a great opportunity to have some fun, learn new skills and share it with the testing community.</p>
<p><a href="/img/2016-07-31/30daystesting.png"><img src="/img/2016-07-31/30daystesting-small.png" alt="30 Days of Testing" /></a></p>
<p>Unfortunately the challenge schedule overlapped with holiday season full of traveling and spending time on summer cottage. Therefore I did cheat a bit regarding the rules of the challenge. Not all challenges were achieved on the given day of the challenge as few challenges was needed to be rescheduled for more suitable days. Also some of the challenges were needed to be skipped due lack of time.</p>
<p>Here is a short diary of what I did - and missed - during the 30 days testing challenge.</p>
<h3 id="1-buy-one-testing-related-book-and-read-it-by-day-30">1 Buy one testing related book and read it by day 30</h3>
<p>Decided to read <a href="https://www.amazon.com/Agile-Testing-Practical-Guide-Testers/dp/0321534468">Agile Testing: A Practical Guide for Testers and Agile Teams</a> by Janet Grecory and Lisa Crispin. I had it already in my bookself waiting to be read, and #30DaysOfTesting gave the needed motivation push for it.</p>
<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">Starting <a href="https://twitter.com/hashtag/30daysoftesting?src=hash">#30daysoftesting</a> with Agile Testing by <a href="https://twitter.com/lisacrispin">@lisacrispin</a> and <a href="https://twitter.com/janetgregoryca">@janetgregoryca</a>. <a href="https://t.co/bTfDETnh6z">https://t.co/bTfDETnh6z</a>.</p>— Juho Perälä (@juperala) <a href="https://twitter.com/juperala/status/748824202812198912">July 1, 2016</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>Great book, still few chapters to finalize…</p>
<h3 id="2-take-a-photo-of-something-you-are-doing-at-work">2 Take a photo of something you are doing at work</h3>
<p>The 2nd day was Saturday, so no work on that day. But read some chapter’s on Agile Testing book. Here’s a pic.</p>
<p><a href="/img/2016-07-31/agiletesting.png"><img src="/img/2016-07-31/agiletesting-small.png" alt="30 Days of Testing" /></a></p>
<h3 id="3-listen-to-a-testing-podcast">3 Listen to a testing podcast</h3>
<p>Listened bunch of podcasts from Joe Colantonio’s #TestTalks channel. As we are taking Appium in use at the current project, I was especially interested on <a href="https://joecolantonio.com/testtalks/107-appium-beginners-james-koch/">Appium for Beginners with James Koch</a>.</p>
<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">3rd day of <a href="https://twitter.com/hashtag/30daysoftesting?src=hash">#30daysoftesting</a>. Now listening <a href="https://twitter.com/hashtag/TestTalks?src=hash">#TestTalks</a> podcast about <a href="https://twitter.com/hashtag/Appium?src=hash">#Appium</a> with <a href="https://twitter.com/jcolantonio">@jcolantonio</a> and James Koch.</p>— Juho Perälä (@juperala) <a href="https://twitter.com/juperala/status/749548989423517696">July 3, 2016</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<h3 id="4-share-a-testing-blog-post-with-a-non-tester">4 Share a testing blog post with a non-tester</h3>
<p>Shared a blog <a href="http://agiletesting.blogspot.fi/2016/06/running-jenkins-jobs-in-docker.html">post</a> about running Jenkins jobs on Docker containers to a colleague.</p>
<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">Read and shared great blog post(s) about running <a href="https://twitter.com/hashtag/Jenkins?src=hash">#Jenkins</a> jobs on <a href="https://twitter.com/hashtag/Docker?src=hash">#Docker</a> containers by <a href="https://twitter.com/griggheo">@griggheo</a>. <a href="https://t.co/EtKDFtkdwB">https://t.co/EtKDFtkdwB</a> <a href="https://twitter.com/hashtag/30daysoftesting?src=hash">#30daysoftesting</a></p>— Juho Perälä (@juperala) <a href="https://twitter.com/juperala/status/750061884074844160">July 4, 2016</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<h3 id="5-read-and-comment-on-one-blog-post">5 Read and comment on one blog post</h3>
<p>Read an excellent blog <a href="http://dev.solita.fi/2016/07/04/10-things-that-make-development-process-awesome.html">post</a> about thing that make development awesome.</p>
<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">Day 5 of <a href="https://twitter.com/hashtag/30daysoftesting?src=hash">#30daysoftesting</a>. Excellent article on things that makes development awesome by <a href="https://twitter.com/jarzka">@jarzka</a> at <a href="https://twitter.com/SolitaOy">@SolitaOy</a>. <a href="https://t.co/zNabHtjXlo">https://t.co/zNabHtjXlo</a></p>— Juho Perälä (@juperala) <a href="https://twitter.com/juperala/status/750278283959345153">July 5, 2016</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<h3 id="6-perform-a-crazy-test">6 Perform a crazy test</h3>
<p>Wrote a RESTfull API test for <a href="https://www.foaas.com/">FOASS (Fuck Off As A Service)</a> service. Service seems to work as expected.</p>
<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">As a crazy test for <a href="https://twitter.com/hashtag/30daysoftesting?src=hash">#30daysoftesting</a> wrote automated test (or check if you prefer) for <a href="https://twitter.com/foaas">@foaas</a> service. Test Pass. <a href="https://t.co/l4ZPMbNN4p">pic.twitter.com/l4ZPMbNN4p</a></p>— Juho Perälä (@juperala) <a href="https://twitter.com/juperala/status/750641236512477184">July 6, 2016</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<h3 id="7-find-an-accessibility-bug">7 Find an accessibility bug</h3>
<p>No findings for the given day, but earlier this month found few issues with our test server accessibility due faulty configurations.</p>
<h3 id="8-download-a-mobile-app-find-5-bugs-and-send-the-feedback-to-the-creator">8 Download a mobile app, find 5 bugs and send the feedback to the creator</h3>
<p>Skipped due great weather, sun and traveling to summer cottage. Same justification applies also to the rest of the skipped challenges. :)</p>
<h3 id="9-create-a-mindmap">9 Create a mindmap</h3>
<p>Created a “holiday inspired” mindmap.</p>
<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">Mindmap for day 9 of <a href="https://twitter.com/hashtag/30daysoftesting?src=hash">#30daysoftesting</a>. <a href="https://t.co/pVmo6Oof3p">pic.twitter.com/pVmo6Oof3p</a></p>— Juho Perälä (@juperala) <a href="https://twitter.com/juperala/status/751700849034981376">July 9, 2016</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<h3 id="10-find-an-event-to-attend-online-or-face-to-face">10 Find an event to attend (online or face to face)</h3>
<p>Participated in Ministry of Testing online masterclass <a href="http://www.ministryoftesting.com/training-events/">How To Build A Regression Checking Strategy with Mark Winteringham</a>.</p>
<h3 id="11-take-a-picture-of-your-team">11 Take a picture of your team</h3>
<p>No team picture due summer vacations, whole team is on holiday. As a replacement here’s a family holiday picture from QStock festival.</p>
<blockquote class="instagram-media" data-instgrm-captioned="" data-instgrm-version="7" style=" background:#FFF; border:0; border-radius:3px; box-shadow:0 0 1px 0 rgba(0,0,0,0.5),0 1px 10px 0 rgba(0,0,0,0.15); margin: 1px; max-width:658px; padding:0; width:99.375%; width:-webkit-calc(100% - 2px); width:calc(100% - 2px);"><div style="padding:8px;"> <div style=" background:#F8F8F8; line-height:0; margin-top:40px; padding:37.5% 0; text-align:center; width:100%;"> <div style=" background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACwAAAAsCAMAAAApWqozAAAABGdBTUEAALGPC/xhBQAAAAFzUkdCAK7OHOkAAAAMUExURczMzPf399fX1+bm5mzY9AMAAADiSURBVDjLvZXbEsMgCES5/P8/t9FuRVCRmU73JWlzosgSIIZURCjo/ad+EQJJB4Hv8BFt+IDpQoCx1wjOSBFhh2XssxEIYn3ulI/6MNReE07UIWJEv8UEOWDS88LY97kqyTliJKKtuYBbruAyVh5wOHiXmpi5we58Ek028czwyuQdLKPG1Bkb4NnM+VeAnfHqn1k4+GPT6uGQcvu2h2OVuIf/gWUFyy8OWEpdyZSa3aVCqpVoVvzZZ2VTnn2wU8qzVjDDetO90GSy9mVLqtgYSy231MxrY6I2gGqjrTY0L8fxCxfCBbhWrsYYAAAAAElFTkSuQmCC); display:block; height:44px; margin:0 auto -44px; position:relative; top:-22px; width:44px;"></div></div> <p style=" margin:8px 0 0 0; padding:0 4px;"> <a href="https://www.instagram.com/p/BIc8r7wAqrm/" style=" color:#000; font-family:Arial,sans-serif; font-size:14px; font-style:normal; font-weight:normal; line-height:17px; text-decoration:none; word-wrap:break-word;" target="_blank">Qstock 2016. #Qstock #Vesala #Oulu</a></p> <p style=" color:#c9c8cd; font-family:Arial,sans-serif; font-size:14px; line-height:17px; margin-bottom:0; margin-top:8px; overflow:hidden; padding:8px 0 7px; text-align:center; text-overflow:ellipsis; white-space:nowrap;">A photo posted by Juho Perälä (@juperala) on <time style=" font-family:Arial,sans-serif; font-size:14px; line-height:17px;" datetime="2016-07-29T16:36:25+00:00">Jul 29, 2016 at 9:36am PDT</time></p></div></blockquote>
<script async="" defer="" src="//platform.instagram.com/en_US/embeds.js"></script>
<h3 id="12-doodle-a-problem">12 Doodle a problem</h3>
<p>Skipped.</p>
<h3 id="13-find-a-user-experience-problem">13 Find a user experience problem</h3>
<p>The <a href="http://onedrive.live.com">Microsoft OneDrive</a> login process seems to make absolutely no sense. To access your OneDrive account, you need to go through four (4) page login process:</p>
<p>On the <strong>1st page</strong> you select to sign-in.
<a href="/img/2016-07-31/onedrive1.png"><img src="/img/2016-07-31/onedrive1-small.png" alt="1st login page" /></a></p>
<p>Next, on the <strong>2nd page</strong> you enter you email address.
<a href="/img/2016-07-31/onedrive2.png"><img src="/img/2016-07-31/onedrive2-small.png" alt="2nd login page" /></a></p>
<p>Then, on the <strong>3rd page</strong> you select either personal or business account.
<a href="/img/2016-07-31/onedrive3.png"><img src="/img/2016-07-31/onedrive3-small.png" alt="3rd login page" /></a></p>
<p>And finally, on the <strong>4th page</strong> you <em>re-enter</em> your email address and password for OneDrive login.
<a href="/img/2016-07-31/onedrive4.png"><img src="/img/2016-07-31/onedrive4-small.png" alt="4th login page" /></a></p>
<p>As a user I would appreciate a faster way to access my OneDrive content via browser.</p>
<h3 id="14-step-outside-of-your-comfort-zone">14 Step outside of your comfort zone</h3>
<p>Got a MacBook Pro for running Appium test on iOS devices/emulator. The OSX user interface is way outside of my comfort zone. Luckily you can do most things remotely via SSH terminal. And it seems I am getting used to the GUI also, pushing out the zone boundaries…</p>
<h3 id="15-find-a-problem-with-an-e-commerce-website">15 Find a problem with an e-commerce website</h3>
<p>Found an issue on missing localization of undefined product category on a online shop. Reported issue.</p>
<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">Day 15 of <a href="https://twitter.com/hashtag/30daysoftesting?src=hash">#30daysoftesting</a>. Found bug in online shop. Missing category localization for products with no category. Shows "root" as category.</p>— Juho Perälä (@juperala) <a href="https://twitter.com/juperala/status/754011543491272706">July 15, 2016</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<h3 id="16-go-to-a-non-testing-event">16 Go to a non-testing event</h3>
<p>Did not manage to find any good professional events to participate for July. Generally July is holiday season in Finland and there is not many work-related events organized. But participated on a bunch of non-professional events such as <a href="http://www.qstock.fi/">QStock</a> music festival.</p>
<h3 id="17-find-and-share-a-quote-that-inspires-you">17 Find and share a quote that inspires you</h3>
<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">“You may be disappointed if you fail, but you are doomed if you don’t try.” -Beverly Sills. <a href="https://twitter.com/hashtag/quoteoftheday?src=hash">#quoteoftheday</a> <a href="https://twitter.com/hashtag/30daysoftesting?src=hash">#30daysoftesting</a></p>— Juho Perälä (@juperala) <a href="https://twitter.com/juperala/status/754613477147369476">July 17, 2016</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<h3 id="18-find-a-broken-link-and-report-it">18 Find a broken link. and report it</h3>
<p>Earlier this month found broken links in test data. Fixed.</p>
<h3 id="19-find-and-use-a-new-tool">19 Find and use a new tool</h3>
<p>During June and July I familiarized with <a href="http://appium.io/">Appium</a> and got mobile browser tests running from Jenkins on iOS and Android devices/emulators. Seems like a very nice framework with smooth transition from Selenium based desktop browser testing.</p>
<p>Also tried out <a href="https://www.getpostman.com/">Postman</a> tool for REST API testing and <a href="https://www.splunk.com/">Splunk</a> tool for real-time server log monitoring.</p>
<h3 id="20-find-a-good-place-to-perform-some-security-tests">20 Find a good place to perform some security tests</h3>
<p>Skipped.</p>
<h3 id="21-pair-test-with-someone">21 Pair test with someone</h3>
<p>Before holidays paired with colleague for testing new product functionality. Something that we do more or less regularly.</p>
<h3 id="22-share-your-favourite-testing-tool">22 Share your favourite testing tool</h3>
<p>Selenium WebDriver. At the moment it is <strong>the tool</strong> for the desktop and mobile web-testing. Widely used and supported by various tools, platforms and programming languages.</p>
<p>Ps. Selenium 3 beta is just <a href="https://seleniumhq.wordpress.com/2016/07/29/announcing-selenium-3-0-beta1/">released</a>.</p>
<h3 id="23-help-someone-test-better">23 Help someone test better</h3>
<p>I try to help out colleagues on daily basis when possible, but no specific activities to be mentioned here. Our team keeps active dialogue to learn from each others and to share the knowledge.</p>
<h3 id="24-connect-with-a-tester-who-you-havent-previously-connected-with">24 Connect with a tester who you haven’t previously connected with</h3>
<p>Participated on testing related discussions on social media with couple of new contacts.</p>
<h3 id="25-contribute-to-a-testing-discussion">25 Contribute to a testing discussion</h3>
<p>See previous.</p>
<h3 id="26-invite-a-non-tester-to-a-test-event">26 Invite a non-tester to a test event</h3>
<p>Sent invitation to <a href="http://testingassembly.fistb.fi/home">FiSTB Testing Assembly 2016</a> for a few colleagues.</p>
<h3 id="27-say-something-nice-about-the-thing-you-just-tested">27 Say something nice about the thing you just tested</h3>
<p>The usability and UX is really nice on the product we are currently developing.</p>
<h3 id="28-summarise-an-issue-in-140-characters-or-less">28 Summarise an issue in 140 characters or less</h3>
<p>Done. Project related issue, not for public sharing.</p>
<h3 id="29-find-an-out-by-one-error">29 Find an out by one error</h3>
<p>Skipped. Did not find any.</p>
<h3 id="30-give-someone-positive-feedback">30 Give someone positive feedback</h3>
<p>I really appreciate the effort my employer and it’s core team put-out to make it great place to work. Love the company culture and can-do-attitude.</p>
<h3 id="31-bonus-share-your-30-day-challenge-experience-on-youtube-instagram-twitter-or-a-blog-post">31 BONUS: Share your 30 day challenge experience on youtube, instagram, twitter or a blog post!</h3>
<p>This post. Done.</p>jperalaMinistry of Testing started a #30DaysOfTesting challenge for the July. I decided to join the challenge as it provided a great opportunity to have some fun, learn new skills and share it with the testing community. Unfortunately the challenge schedule overlapped with holiday season full of traveling and spending time on summer cottage. Therefore I did cheat a bit regarding the rules of the challenge. Not all challenges were achieved on the given day of the challenge as few challenges was needed to be rescheduled for more suitable days. Also some of the challenges were needed to be skipped due lack of time. Here is a short diary of what I did - and missed - during the 30 days testing challenge. 1 Buy one testing related book and read it by day 30 Decided to read Agile Testing: A Practical Guide for Testers and Agile Teams by Janet Grecory and Lisa Crispin. I had it already in my bookself waiting to be read, and #30DaysOfTesting gave the needed motivation push for it. Starting #30daysoftesting with Agile Testing by @lisacrispin and @janetgregoryca. https://t.co/bTfDETnh6z.— Juho Perälä (@juperala) July 1, 2016 Great book, still few chapters to finalize… 2 Take a photo of something you are doing at work The 2nd day was Saturday, so no work on that day. But read some chapter’s on Agile Testing book. Here’s a pic. 3 Listen to a testing podcast Listened bunch of podcasts from Joe Colantonio’s #TestTalks channel. As we are taking Appium in use at the current project, I was especially interested on Appium for Beginners with James Koch. 3rd day of #30daysoftesting. Now listening #TestTalks podcast about #Appium with @jcolantonio and James Koch.— Juho Perälä (@juperala) July 3, 2016 4 Share a testing blog post with a non-tester Shared a blog post about running Jenkins jobs on Docker containers to a colleague. Read and shared great blog post(s) about running #Jenkins jobs on #Docker containers by @griggheo. https://t.co/EtKDFtkdwB #30daysoftesting— Juho Perälä (@juperala) July 4, 2016 5 Read and comment on one blog post Read an excellent blog post about thing that make development awesome. Day 5 of #30daysoftesting. Excellent article on things that makes development awesome by @jarzka at @SolitaOy. https://t.co/zNabHtjXlo— Juho Perälä (@juperala) July 5, 2016 6 Perform a crazy test Wrote a RESTfull API test for FOASS (Fuck Off As A Service) service. Service seems to work as expected. As a crazy test for #30daysoftesting wrote automated test (or check if you prefer) for @foaas service. Test Pass. pic.twitter.com/l4ZPMbNN4p— Juho Perälä (@juperala) July 6, 2016 7 Find an accessibility bug No findings for the given day, but earlier this month found few issues with our test server accessibility due faulty configurations. 8 Download a mobile app, find 5 bugs and send the feedback to the creator Skipped due great weather, sun and traveling to summer cottage. Same justification applies also to the rest of the skipped challenges. :) 9 Create a mindmap Created a “holiday inspired” mindmap. Mindmap for day 9 of #30daysoftesting. pic.twitter.com/pVmo6Oof3p— Juho Perälä (@juperala) July 9, 2016 10 Find an event to attend (online or face to face) Participated in Ministry of Testing online masterclass How To Build A Regression Checking Strategy with Mark Winteringham. 11 Take a picture of your team No team picture due summer vacations, whole team is on holiday. As a replacement here’s a family holiday picture from QStock festival. Qstock 2016. #Qstock #Vesala #Oulu A photo posted by Juho Perälä (@juperala) on Jul 29, 2016 at 9:36am PDT 12 Doodle a problem Skipped. 13 Find a user experience problem The Microsoft OneDrive login process seems to make absolutely no sense. To access your OneDrive account, you need to go through four (4) page login process: On the 1st page you select to sign-in. Next, on the 2nd page you enter you email address. Then, on the 3rd page you select either personal or business account. And finally, on the 4th page you re-enter your email address and password for OneDrive login. As a user I would appreciate a faster way to access my OneDrive content via browser. 14 Step outside of your comfort zone Got a MacBook Pro for running Appium test on iOS devices/emulator. The OSX user interface is way outside of my comfort zone. Luckily you can do most things remotely via SSH terminal. And it seems I am getting used to the GUI also, pushing out the zone boundaries… 15 Find a problem with an e-commerce website Found an issue on missing localization of undefined product category on a online shop. Reported issue. Day 15 of #30daysoftesting. Found bug in online shop. Missing category localization for products with no category. Shows "root" as category.— Juho Perälä (@juperala) July 15, 2016 16 Go to a non-testing event Did not manage to find any good professional events to participate for July. Generally July is holiday season in Finland and there is not many work-related events organized. But participated on a bunch of non-professional events such as QStock music festival. 17 Find and share a quote that inspires you “You may be disappointed if you fail, but you are doomed if you don’t try.” -Beverly Sills. #quoteoftheday #30daysoftesting— Juho Perälä (@juperala) July 17, 2016 18 Find a broken link. and report it Earlier this month found broken links in test data. Fixed. 19 Find and use a new tool During June and July I familiarized with Appium and got mobile browser tests running from Jenkins on iOS and Android devices/emulators. Seems like a very nice framework with smooth transition from Selenium based desktop browser testing. Also tried out Postman tool for REST API testing and Splunk tool for real-time server log monitoring. 20 Find a good place to perform some security tests Skipped. 21 Pair test with someone Before holidays paired with colleague for testing new product functionality. Something that we do more or less regularly. 22 Share your favourite testing tool Selenium WebDriver. At the moment it is the tool for the desktop and mobile web-testing. Widely used and supported by various tools, platforms and programming languages. Ps. Selenium 3 beta is just released. 23 Help someone test better I try to help out colleagues on daily basis when possible, but no specific activities to be mentioned here. Our team keeps active dialogue to learn from each others and to share the knowledge. 24 Connect with a tester who you haven’t previously connected with Participated on testing related discussions on social media with couple of new contacts. 25 Contribute to a testing discussion See previous. 26 Invite a non-tester to a test event Sent invitation to FiSTB Testing Assembly 2016 for a few colleagues. 27 Say something nice about the thing you just tested The usability and UX is really nice on the product we are currently developing. 28 Summarise an issue in 140 characters or less Done. Project related issue, not for public sharing. 29 Find an out by one error Skipped. Did not find any. 30 Give someone positive feedback I really appreciate the effort my employer and it’s core team put-out to make it great place to work. Love the company culture and can-do-attitude. 31 BONUS: Share your 30 day challenge experience on youtube, instagram, twitter or a blog post! This post. Done.Adding comments to static site2016-06-26T22:55:00+03:002020-03-26T17:11:00+02:00https://jperala.fi/2016/06/26/adding-comments-to-static-site<p>As I wrote <a href="/2016/02/28/going-static/">earlier</a>, I migrated my blog from Wordpress to static Jekyll website. I really like the static website as a concept as it makes the site maintenance less complicated and straightforward. However, being a static site, there is no native support for adding dynamic features such as post commenting.</p>
<p>Luckily, there is easy way to add commenting to static sites using external services such as <a href="https://disqus.com/">Disqus</a>.</p>
<p>Here’s a step-by-step guide for adding Disqus commenting on Jekyll site.</p>
<h1 id="create-disqus-account">Create Disqus account</h1>
<p>Register to <a href="https://disqus.com/">Disqus</a> and add your first website. Disqus will ask a shortname for your site and opens the site settings page. Enter site URL and name.</p>
<p><a href="/img/2016-06-26/disqus.png"><img src="/img/2016-06-26/disqus-small.png" alt="Disqus site configuration" /></a></p>
<p>Next go to Disqus installation page and copy <em>Universal Code</em> snippet to text editor.</p>
<figure class="highlight"><pre><code class="language-js" data-lang="js"><span class="o"><</span><span class="nx">div</span> <span class="nx">id</span><span class="o">=</span><span class="s2">"disqus_thread"</span><span class="o">><</span><span class="sr">/div></span><span class="err">
</span><span class="o"><</span><span class="nx">script</span><span class="o">></span>
<span class="cm">/**
* RECOMMENDED CONFIGURATION VARIABLES: EDIT AND UNCOMMENT THE SECTION BELOW TO INSERT DYNAMIC VALUES FROM YOUR PLATFORM OR CMS.
* LEARN WHY DEFINING THESE VARIABLES IS IMPORTANT: https://disqus.com/admin/universalcode/#configuration-variables
*/</span>
<span class="cm">/*
var disqus_config = function () {
this.page.url = PAGE_URL; // Replace PAGE_URL with your page's canonical URL variable
this.page.identifier = PAGE_IDENTIFIER; // Replace PAGE_IDENTIFIER with your page's unique identifier variable
};
*/</span>
<span class="p">(</span><span class="kd">function</span><span class="p">()</span> <span class="p">{</span> <span class="c1">// DON'T EDIT BELOW THIS LINE
</span>
<span class="kd">var</span> <span class="nx">d</span> <span class="o">=</span> <span class="nb">document</span><span class="p">,</span> <span class="nx">s</span> <span class="o">=</span> <span class="nx">d</span><span class="p">.</span><span class="nx">createElement</span><span class="p">(</span><span class="s1">'script'</span><span class="p">);</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">src</span> <span class="o">=</span> <span class="s1">'//SITE_SHORTNAME.disqus.com/embed.js'</span><span class="p">;</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">setAttribute</span><span class="p">(</span><span class="s1">'data-timestamp'</span><span class="p">,</span> <span class="o">+</span><span class="k">new</span> <span class="nb">Date</span><span class="p">());</span>
<span class="p">(</span><span class="nx">d</span><span class="p">.</span><span class="nx">head</span> <span class="o">||</span> <span class="nx">d</span><span class="p">.</span><span class="nx">body</span><span class="p">).</span><span class="nx">appendChild</span><span class="p">(</span><span class="nx">s</span><span class="p">);</span>
<span class="p">})();</span>
<span class="o"><</span><span class="sr">/script></span><span class="err">
</span><span class="o"><</span><span class="nx">noscript</span><span class="o">></span><span class="nx">Please</span> <span class="nx">enable</span> <span class="nx">JavaScript</span> <span class="nx">to</span> <span class="nx">view</span> <span class="nx">the</span> <span class="o"><</span><span class="nx">a</span> <span class="nx">href</span><span class="o">=</span><span class="s2">"https://disqus.com/?ref_noscript"</span> <span class="nx">rel</span><span class="o">=</span><span class="s2">"nofollow"</span><span class="o">></span><span class="nx">comments</span> <span class="nx">powered</span> <span class="nx">by</span> <span class="nx">Disqus</span><span class="p">.</span><span class="o"><</span><span class="sr">/a></</span><span class="nx">noscript</span><span class="o">></span></code></pre></figure>
<p>All done on Disqus, next we need to edit Jekyll template to enable commenting.</p>
<h1 id="modify-jekyll-post-template">Modify Jekyll post template</h1>
<p>Open the appropriate post template from your Jekyll project. On the <em>Clean Blog</em> theme the post template file is <em>/_layouts/post.html</em>. Add the copied <em>Universal Code</em> snipped inside <em>{% if page.comments %}</em> and <em>{% endif %}</em> liquid tags.</p>
<p>Uncomment the recommended configurations and update variables so that the correct information is picked up from Jekyll configuration:</p>
<figure class="highlight"><pre><code class="language-js" data-lang="js"><span class="kd">var</span> <span class="nx">disqus_config</span> <span class="o">=</span> <span class="kd">function</span> <span class="p">()</span> <span class="p">{</span>
<span class="k">this</span><span class="p">.</span><span class="nx">page</span><span class="p">.</span><span class="nx">url</span> <span class="o">=</span> <span class="s2">"{{ site.url }}{{ page.url }}"</span><span class="p">;</span>
<span class="k">this</span><span class="p">.</span><span class="nx">page</span><span class="p">.</span><span class="nx">identifier</span> <span class="o">=</span> <span class="s2">"{{ page.id }}"</span><span class="p">;</span>
<span class="p">};</span></code></pre></figure>
<p>All done here, now Jekyll is ready to show comments for posts where commenting is enabled.</p>
<h1 id="enabled-comments-on-posts">Enabled comments on posts</h1>
<p>Now all you need to do is to go through your post markdown files, and add <em>“comments: true”</em> into the post <a href="https://jekyllrb.com/docs/frontmatter/">FrontMatter</a> configuration headers.</p>
<p>Once done, reload pages, and Disqus commenting will be enabled.</p>jperalaAs I wrote earlier, I migrated my blog from Wordpress to static Jekyll website. I really like the static website as a concept as it makes the site maintenance less complicated and straightforward. However, being a static site, there is no native support for adding dynamic features such as post commenting. Luckily, there is easy way to add commenting to static sites using external services such as Disqus. Here’s a step-by-step guide for adding Disqus commenting on Jekyll site. Create Disqus account Register to Disqus and add your first website. Disqus will ask a shortname for your site and opens the site settings page. Enter site URL and name. Next go to Disqus installation page and copy Universal Code snippet to text editor. <div id="disqus_thread"></div> <script> /** * RECOMMENDED CONFIGURATION VARIABLES: EDIT AND UNCOMMENT THE SECTION BELOW TO INSERT DYNAMIC VALUES FROM YOUR PLATFORM OR CMS. * LEARN WHY DEFINING THESE VARIABLES IS IMPORTANT: https://disqus.com/admin/universalcode/#configuration-variables */ /* var disqus_config = function () { this.page.url = PAGE_URL; // Replace PAGE_URL with your page's canonical URL variable this.page.identifier = PAGE_IDENTIFIER; // Replace PAGE_IDENTIFIER with your page's unique identifier variable }; */ (function() { // DON'T EDIT BELOW THIS LINE var d = document, s = d.createElement('script'); s.src = '//SITE_SHORTNAME.disqus.com/embed.js'; s.setAttribute('data-timestamp', +new Date()); (d.head || d.body).appendChild(s); })(); </script> <noscript>Please enable JavaScript to view the <a href="https://disqus.com/?ref_noscript" rel="nofollow">comments powered by Disqus.</a></noscript> All done on Disqus, next we need to edit Jekyll template to enable commenting. Modify Jekyll post template Open the appropriate post template from your Jekyll project. On the Clean Blog theme the post template file is /_layouts/post.html. Add the copied Universal Code snipped inside {% if page.comments %} and {% endif %} liquid tags. Uncomment the recommended configurations and update variables so that the correct information is picked up from Jekyll configuration: var disqus_config = function () { this.page.url = "{{ site.url }}{{ page.url }}"; this.page.identifier = "{{ page.id }}"; }; All done here, now Jekyll is ready to show comments for posts where commenting is enabled. Enabled comments on posts Now all you need to do is to go through your post markdown files, and add “comments: true” into the post FrontMatter configuration headers. Once done, reload pages, and Disqus commenting will be enabled.Half dozen of hosted CI platforms2016-06-11T11:23:00+03:002020-03-26T17:10:00+02:00https://jperala.fi/2016/06/11/hosted-ci-overview<p>Continuous integration (CI) and Continous Delivery (CD) pipeline is one of the key enablers for agile development, release and continuous quality assurance. Traditionally CI platforms, like <a href="https://jenkins.io/">Jenkins</a>, <a href="https://www.jetbrains.com/teamcity/">Teamcity</a> and <a href="https://www.atlassian.com/software/bamboo">Atlassian Bamboo</a>, have been self-hosted server applications deployed in in-house infrastructures.</p>
<p>During the last years dozens of SaaS-based hosted CI platforms have emerged. Having hosted CI environments reduces the need for environment maintenance and enables easy scaling of build and test resources with cost-effective pricing plans. Although many large enterprises still prefer to host their own CI infrastructures, often due security and legacy reasons, the hosted CI platforms provide good alternative for various needs from personal and open source projects to business and enterprise customers.</p>
<p>Another change in the CI domain is change towards automated infrastructure and build configuration. In traditional CI environments build pipelines were configured manually via user interface, whereas the latest platforms support <a href="https://en.wikipedia.org/wiki/Infrastructure_as_Code">Infrastructure as Code</a> paradigm by allowing configuration of build pipelines via configuration files committed in the project repositories. In addition, most platforms support RESTfull APIs and web hooks enabling integration of CI platforms with 3rd party services for configuration and monitoring of the build executions.</p>
<p>As a long-time user of Jenkins and Bamboo CI tools I wanted to catch up with latest hosted alternatives and their pricing and capabilities. To get an overview of the setup process and capabilities, I configured example Java/Maven project with Selenium browser tests for execution. In this post you find a short overview of reviewed tools and initial observations made during the project setup and configuration.</p>
<h2 id="travisci">TravisCI</h2>
<p>TravisCI is probably the most popular of the hosted CI platforms - especially in Open Source community with over 300k projects relying on it. It supports building for both Linux and OSX environments. The big limitation of TravisCI is integration only with GitHub repositories.</p>
<p>The TravisCI setup for basic Java builds with browser testing is straightforward. Login to TravisCI with GitHub account, select repository to follow, push <em>.travis.yml</em> configuration into repository and TravisCI will pick the changes and trigger builds automatically. TravisCI environment has Firefox preinstalled and only Xvfb virtual frame buffer needs to be configured for using Firefox. The example <em>.travis.yml</em> file is presented below:</p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="na">language</span><span class="pi">:</span> <span class="s">java</span><span class="err">
</span>
<span class="na">jdk</span><span class="pi">:</span> <span class="s">oraclejdk8</span><span class="err">
</span>
<span class="na">before_script</span><span class="pi">:</span><span class="err">
</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">export</span><span class="nv"> </span><span class="s">DISPLAY=:99.0"</span><span class="err">
</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">sh</span><span class="nv"> </span><span class="s">-e</span><span class="nv"> </span><span class="s">/etc/init.d/xvfb</span><span class="nv"> </span><span class="s">start"</span><span class="err">
</span>
<span class="pi">-</span> <span class="s">sleep 3</span> <span class="c1"># give xvfb some time to start
</span>
<span class="na">script</span><span class="pi">:</span> <span class="s">mvn clean verify</span></code></pre></figure>
<p>Being free for Open Source project the TravisCI is very compelling alternative for public repositories, but for personal use the $129/month Startup plan feels overpriced and cheaper personal price plan would be welcomed.</p>
<ul>
<li><strong>Homepage:</strong> <a href="https://travis-ci.com">https://travis-ci.com</a></li>
<li><strong>Platforms</strong>: Linux, OS X</li>
<li><strong>VCS support:</strong> GitHub</li>
<li><strong>Pipeline configuration:</strong> YAML</li>
<li><strong>Pricing:</strong>
<ul>
<li>Free for Open Source (unlimited build minutes and repositories)</li>
<li>Starting at $129/month for private projects (2 concurrent jobs, unlimited build minutes and repositories)</li>
<li>Free trial (first 100 private builds free)</li>
<li>Student plan (100 free builds per month)</li>
</ul>
</li>
<li><strong>On-Site deployment:</strong> Yes. TravisCI Enterprise.</li>
<li><strong>Maturity:</strong>
<ul>
<li>Founded 2011</li>
<li>Over 235k user accounts</li>
<li>Over 300k open source projects</li>
<li>References: Facebook, Heroku, Mozilla, Twitter,…</li>
</ul>
</li>
<li><strong>API:</strong> REST</li>
<li><strong>Browser testing:</strong> Firefox, PhantomJS</li>
<li><strong>Other:</strong>
<ul>
<li>iOS and Android (Beta) SDKs</li>
</ul>
</li>
</ul>
<h2 id="circleci">CircleCI</h2>
<p>CircleCI is also one of the well-established hosted CI platforms. Similarly with TravisCI the CircleCI supports only GitHub repositories. The pricing plans differ from TravisCI as CircleCI offers each user first Linux container free of charge (1500 build minutes / month).</p>
<p>The usage of CircleCI is very similar with TravisCI. Login with GitHub account, select repository, add <em>circle.yml</em> configuration, and CircleCI will pick up the changes and trigger builds automatically. The only configuration needed was selection of correct JDK version and no additional configuration were needed for running Selenium tests with Firefox. The example <em>circle.yml</em> configuration is presented below:</p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="na">machine</span><span class="pi">:</span><span class="err">
</span>
<span class="na">java</span><span class="pi">:</span><span class="err">
</span>
<span class="na">version</span><span class="pi">:</span> <span class="s">oraclejdk8</span></code></pre></figure>
<p>With initial free Linux container the CircleCI provides great choice for personal use and can be scaled up for professional and business use.</p>
<ul>
<li><strong>Homepage:</strong> <a href="https://circleci.com/">https://circleci.com</a></li>
<li><strong>Platforms</strong>: Linux, OS X</li>
<li><strong>VCS support:</strong> GitHub</li>
<li><strong>Pipeline configuration:</strong> YAML</li>
<li><strong>On-site deployment:</strong> Yes. CircleCI Enterprise.</li>
<li><strong>Pricing:</strong>
<ul>
<li>First Linux container free (unlimited repos and users, 1500 build minutes/month)</li>
<li>Additional containers $50/each, OSX plans starting from $39/month.</li>
<li>Four free Linux containers for Open Source projects.</li>
</ul>
</li>
<li><strong>Maturity:</strong>
<ul>
<li>Founded 2011</li>
<li>Over 90k user accounts</li>
<li>References: Facebook, Spotify, Docker, GoPro,…</li>
</ul>
</li>
<li><strong>API:</strong> REST</li>
<li><strong>Browser:</strong> Firefox, Chrome, PhantomJS</li>
<li><strong>Other:</strong>
<ul>
<li>SSH access for debugging builds.</li>
<li>iOS and Android SDKs</li>
</ul>
</li>
</ul>
<h2 id="snapci">SnapCI:</h2>
<p>SnapCI differs from TravisCI and CircleCI as it does not support use of configuration files and build configurations are carried out via interactive user interface. The SnapCI divides build pipeline into consequent stages and each stage is executed in fresh build machine. Also SnapCI supports only GitHub repositories.</p>
<p>Configuration of SnapCI was very easy. After logging in with GitHub account and selecting the repository, the SnapCI preconfigured the build correctly for execution based on the project Maven POM configuration. The SnapCI is good alternative for people looking hosted CI platform with manually managed build configurations.</p>
<ul>
<li><strong>Homepage:</strong> <a href="https://snap-ci.com/">https://snap-ci.com/</a></li>
<li><strong>Platforms</strong>: Linux</li>
<li><strong>VCS support:</strong> GitHub</li>
<li><strong>Pipeline configuration:</strong> No</li>
<li><strong>Pricing:</strong>
<ul>
<li>Free for Open Source (1 parallel build)</li>
<li>Starting at $30/month for private projects (5 private repositories, 1 concurrent build)</li>
</ul>
</li>
<li><strong>On-Site:</strong> No. Only hosting plans.</li>
<li><strong>Maturity:</strong>
<ul>
<li>Founded 2013</li>
<li>References: Mercedes Benz, Applauze, Neo4J, Sky,…</li>
</ul>
</li>
<li><strong>API:</strong> REST</li>
<li><strong>Browser testing:</strong> Firefox, Chrome, PhantomJs</li>
<li><strong>Other:</strong>
<ul>
<li>Android SDKs</li>
</ul>
</li>
</ul>
<h2 id="shippable">Shippable:</h2>
<p>In contrast to previous platforms, Shippable supports both GitHub and BitBucket repositories. The initial plan for first pipeline is free, and additional pipelines can be added with cost-effective pricing.</p>
<p>Shippable claims to supports TravisCI compliant configurations (either named as <em>.travis.yml</em> or <em>shippable.yml</em>), but build failed when using earlier defined configurations for TravisCI. Shippable requires additional definition of Firefox addon for getting the Firefox installed in the container. In addition alternative configuration for setting up the Xvfb frame buffer was required to get build successfully compiled.</p>
<p>Personally I experienced the Shippable user interface to be less intuitive when compared to previously reviewed tools. As a GitHub user I did prefer earlier reviewed tools over Shippable. However, for BitBucket users the Shippable may be a competitive alternative. Also the pricing plans are quite competitive.</p>
<ul>
<li><strong>Homepage:</strong> <a href="https://app.shippable.com">https://app.shippable.com</a></li>
<li><strong>Platforms</strong>: Linux</li>
<li><strong>VCS support:</strong> GitHub, Bitbucket</li>
<li><strong>Pipeline configuration:</strong> YAML</li>
<li><strong>Pricing:</strong>
<ul>
<li>Free for first pipeline (1 build container, 1 pipeline)</li>
<li>Additional build containers $10/each</li>
<li>Additional pipelines $10 for 3 additional pipelines</li>
</ul>
</li>
<li><strong>On-Site:</strong> No</li>
<li><strong>Maturity:</strong>
<ul>
<li>Founded 2013</li>
<li>Over 50k user accounts</li>
<li>References: SAP, Cisco, Postman,…</li>
</ul>
</li>
<li><strong>API:</strong> REST</li>
<li><strong>Browser:</strong> Firefox, PhantomJS</li>
</ul>
<h2 id="codeship">Codeship:</h2>
<p>Similarly with Shippable, the Codeship provides support for both GitHub and BitBucket repositories. The Codeship builds are configured manually via user interface which is quite simple to use but not the most modern. For example, selection of repository is given by entering GitHub repository URL whereas previously reviewed tools provided full GitHub integration with capability of selection of repositories from a list view. The basic Codeship builds do not currently support YAML-based configurations, but this capability has been added to Docker-based containers (CodeJet).</p>
<p>For basic build the only configurations needed were selection of the JDK version using <em>jdk_switcher</em> command and definition of standard Maven goals. Firefox browser tests worked without any addition configurations. In general everything worked as expected, but the service would benefit from YAML based configurations.</p>
<ul>
<li><strong>Homepage:</strong> <a href="https://app.shippable.com">https://app.shippable.com</a></li>
<li><strong>Platforms</strong>: Linux</li>
<li><strong>VCS support:</strong> GitHub, Bitbucket</li>
<li><strong>Pipeline configuration:</strong> No (Currently YAML support only for Docker-based CodeJet)</li>
<li><strong>Pricing:</strong>
<ul>
<li>Free for 1 concurrent build and 1 parallel pipeline (1 build container, 100 builds, 5 projects, unlimited teams)</li>
<li>CodeJet with Docker containers starting from $75/month</li>
</ul>
</li>
<li><strong>On-Site:</strong> No</li>
<li><strong>Maturity:</strong>
<ul>
<li>Founded 2011</li>
<li>Over 75k user accounts</li>
<li>References: Sears, CareerBuilder, BlastWorks,…</li>
</ul>
</li>
<li><strong>API:</strong> REST</li>
<li><strong>Browser testing:</strong> Firefox, Chrome, PhantomJs, CasperJs, SlimerJs</li>
<li><strong>Other:</strong>
<ul>
<li>SSH access for debugging builds.</li>
</ul>
</li>
</ul>
<h2 id="solanoci">SolanoCI:</h2>
<p>The advantage of SolanoCI is it’s wide support for different repositories and version control systems. Whereas all previously reviewed CI tools were limited to GitHub and/or Bitbucket repositories, SolanoCI provides GitHub, GitLab, BitBucket, Stash, Git and Mercurial support.</p>
<p>The build configurations for SolanoCI are made using <em>solano.yml</em> configuration file and configuration was easy to setup. The user interface was not the easiest to use and there seemed to be minor bugs related to accessing build logs. The example <em>solano.yml</em> configuration is show below:</p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="na">java</span><span class="pi">:</span><span class="err">
</span>
<span class="na">java_version</span><span class="pi">:</span> <span class="s">java-8-oracle</span><span class="err">
</span>
<span class="na">maven_version</span><span class="pi">:</span> <span class="s1">'</span><span class="s">3.2.3'</span><span class="err">
</span>
<span class="na">firefox</span><span class="pi">:</span> <span class="s1">'</span><span class="s">27.0.1'</span><span class="err">
</span>
<span class="na">tests</span><span class="pi">:</span><span class="err">
</span>
<span class="pi">-</span> <span class="s">mvn clean verify</span></code></pre></figure>
<hr />
<ul>
<li><strong>Homepage:</strong> <a href="https://www.solanolabs.com">https://www.solanolabs.com</a></li>
<li><strong>VCS support:</strong> Git, GitHub, BitBucket, Stash, GitLab, Mercurial</li>
<li><strong>Pipeline configuration:</strong> YAML</li>
<li><strong>Pricing:</strong>
<ul>
<li>Starting at $15/month (2 workers, 10 worker hours)</li>
<li>14-day free trial available</li>
</ul>
</li>
<li><strong>On-Site deployment:</strong> Yes. Solano Private CI</li>
<li><strong>Maturity:</strong>
<ul>
<li>Founded 2011</li>
<li>Customer references: AirBnB, New Relic,…</li>
</ul>
</li>
<li><strong>API:</strong> REST</li>
<li><strong>Browser testing support:</strong> Firefox, Chrome</li>
<li><strong>Other:</strong>
<ul>
<li>SSH Debug Console (in Private release)</li>
<li>Docker container support</li>
<li>Android SDK</li>
</ul>
</li>
</ul>
<h1 id="and-the-long-list-of-alternatives">And the long list of alternatives</h1>
<p>I limited my review to these six hosted CI platforms, but there is still at least dozen other popular alternatives providing similar features and capabilities, such as:</p>
<ul>
<li><strong><a href="https://drone.io">Drone.io</a></strong>
<ul>
<li>YAML configuration</li>
<li>GitHub and BitBucket repositories</li>
</ul>
</li>
<li><strong><a href="https://www.appveyor.com">AppVeyor</a></strong>
<ul>
<li>Continuous delivery for Windows applications</li>
<li>YAML configuration</li>
<li>GitHub, BitBucket, Visual Studio Online, GitLab, Kiln, Subversion, Git, Mercurial and Stash repositories</li>
</ul>
</li>
<li><strong><a href="http://wercker.com">Wercker</a></strong>
<ul>
<li>YAML configuration</li>
<li>Container-centric platform for automation</li>
</ul>
</li>
<li><strong><a href="https://hosted-ci.com/">HostedCI</a></strong>
<ul>
<li>Jenkins based hosted CI for iOS and OS X projects</li>
<li>Full Git, Mercurial and SVN repository support</li>
</ul>
</li>
<li><strong><a href="https://semaphoreci.com">SemaphoreCI</a></strong>
<ul>
<li>YAML configuration</li>
<li>Docker optimized CI platform</li>
</ul>
</li>
<li><strong><a href="https://www.bitrise.io/">Bitrise</a></strong>
<ul>
<li>CI for mobile platforms (iOS, Android, Xamarin)</li>
<li>YAML configuration</li>
<li>Run locally with CLI</li>
</ul>
</li>
</ul>
<h1 id="summary">Summary</h1>
<p>Nowadays there is available dozens of alternative hosted CI platforms. Most of there are capable of providing support for complex build pipeline configurations, parallelization and integrations support with 3rd party services. As the pricing and features vary, the selection of the platform should be made based on the requirements set for the project.</p>
<p>Based on this short review, I personally liked usability of the TravisCI and CircleCI. Both of these provided very similar approach for configuring and managing build pipelines using YAML configuration files. In addition, the SnapCI provided very clean and intuitive user interface to configure builds via web-based user interface.</p>jperalaContinuous integration (CI) and Continous Delivery (CD) pipeline is one of the key enablers for agile development, release and continuous quality assurance. Traditionally CI platforms, like Jenkins, Teamcity and Atlassian Bamboo, have been self-hosted server applications deployed in in-house infrastructures. During the last years dozens of SaaS-based hosted CI platforms have emerged. Having hosted CI environments reduces the need for environment maintenance and enables easy scaling of build and test resources with cost-effective pricing plans. Although many large enterprises still prefer to host their own CI infrastructures, often due security and legacy reasons, the hosted CI platforms provide good alternative for various needs from personal and open source projects to business and enterprise customers. Another change in the CI domain is change towards automated infrastructure and build configuration. In traditional CI environments build pipelines were configured manually via user interface, whereas the latest platforms support Infrastructure as Code paradigm by allowing configuration of build pipelines via configuration files committed in the project repositories. In addition, most platforms support RESTfull APIs and web hooks enabling integration of CI platforms with 3rd party services for configuration and monitoring of the build executions. As a long-time user of Jenkins and Bamboo CI tools I wanted to catch up with latest hosted alternatives and their pricing and capabilities. To get an overview of the setup process and capabilities, I configured example Java/Maven project with Selenium browser tests for execution. In this post you find a short overview of reviewed tools and initial observations made during the project setup and configuration. TravisCI TravisCI is probably the most popular of the hosted CI platforms - especially in Open Source community with over 300k projects relying on it. It supports building for both Linux and OSX environments. The big limitation of TravisCI is integration only with GitHub repositories. The TravisCI setup for basic Java builds with browser testing is straightforward. Login to TravisCI with GitHub account, select repository to follow, push .travis.yml configuration into repository and TravisCI will pick the changes and trigger builds automatically. TravisCI environment has Firefox preinstalled and only Xvfb virtual frame buffer needs to be configured for using Firefox. The example .travis.yml file is presented below: language: java jdk: oraclejdk8 before_script: - "export DISPLAY=:99.0" - "sh -e /etc/init.d/xvfb start" - sleep 3 # give xvfb some time to start script: mvn clean verify Being free for Open Source project the TravisCI is very compelling alternative for public repositories, but for personal use the $129/month Startup plan feels overpriced and cheaper personal price plan would be welcomed. Homepage: https://travis-ci.com Platforms: Linux, OS X VCS support: GitHub Pipeline configuration: YAML Pricing: Free for Open Source (unlimited build minutes and repositories) Starting at $129/month for private projects (2 concurrent jobs, unlimited build minutes and repositories) Free trial (first 100 private builds free) Student plan (100 free builds per month) On-Site deployment: Yes. TravisCI Enterprise. Maturity: Founded 2011 Over 235k user accounts Over 300k open source projects References: Facebook, Heroku, Mozilla, Twitter,… API: REST Browser testing: Firefox, PhantomJS Other: iOS and Android (Beta) SDKs CircleCI CircleCI is also one of the well-established hosted CI platforms. Similarly with TravisCI the CircleCI supports only GitHub repositories. The pricing plans differ from TravisCI as CircleCI offers each user first Linux container free of charge (1500 build minutes / month). The usage of CircleCI is very similar with TravisCI. Login with GitHub account, select repository, add circle.yml configuration, and CircleCI will pick up the changes and trigger builds automatically. The only configuration needed was selection of correct JDK version and no additional configuration were needed for running Selenium tests with Firefox. The example circle.yml configuration is presented below: machine: java: version: oraclejdk8 With initial free Linux container the CircleCI provides great choice for personal use and can be scaled up for professional and business use. Homepage: https://circleci.com Platforms: Linux, OS X VCS support: GitHub Pipeline configuration: YAML On-site deployment: Yes. CircleCI Enterprise. Pricing: First Linux container free (unlimited repos and users, 1500 build minutes/month) Additional containers $50/each, OSX plans starting from $39/month. Four free Linux containers for Open Source projects. Maturity: Founded 2011 Over 90k user accounts References: Facebook, Spotify, Docker, GoPro,… API: REST Browser: Firefox, Chrome, PhantomJS Other: SSH access for debugging builds. iOS and Android SDKs SnapCI: SnapCI differs from TravisCI and CircleCI as it does not support use of configuration files and build configurations are carried out via interactive user interface. The SnapCI divides build pipeline into consequent stages and each stage is executed in fresh build machine. Also SnapCI supports only GitHub repositories. Configuration of SnapCI was very easy. After logging in with GitHub account and selecting the repository, the SnapCI preconfigured the build correctly for execution based on the project Maven POM configuration. The SnapCI is good alternative for people looking hosted CI platform with manually managed build configurations. Homepage: https://snap-ci.com/ Platforms: Linux VCS support: GitHub Pipeline configuration: No Pricing: Free for Open Source (1 parallel build) Starting at $30/month for private projects (5 private repositories, 1 concurrent build) On-Site: No. Only hosting plans. Maturity: Founded 2013 References: Mercedes Benz, Applauze, Neo4J, Sky,… API: REST Browser testing: Firefox, Chrome, PhantomJs Other: Android SDKs Shippable: In contrast to previous platforms, Shippable supports both GitHub and BitBucket repositories. The initial plan for first pipeline is free, and additional pipelines can be added with cost-effective pricing. Shippable claims to supports TravisCI compliant configurations (either named as .travis.yml or shippable.yml), but build failed when using earlier defined configurations for TravisCI. Shippable requires additional definition of Firefox addon for getting the Firefox installed in the container. In addition alternative configuration for setting up the Xvfb frame buffer was required to get build successfully compiled. Personally I experienced the Shippable user interface to be less intuitive when compared to previously reviewed tools. As a GitHub user I did prefer earlier reviewed tools over Shippable. However, for BitBucket users the Shippable may be a competitive alternative. Also the pricing plans are quite competitive. Homepage: https://app.shippable.com Platforms: Linux VCS support: GitHub, Bitbucket Pipeline configuration: YAML Pricing: Free for first pipeline (1 build container, 1 pipeline) Additional build containers $10/each Additional pipelines $10 for 3 additional pipelines On-Site: No Maturity: Founded 2013 Over 50k user accounts References: SAP, Cisco, Postman,… API: REST Browser: Firefox, PhantomJS Codeship: Similarly with Shippable, the Codeship provides support for both GitHub and BitBucket repositories. The Codeship builds are configured manually via user interface which is quite simple to use but not the most modern. For example, selection of repository is given by entering GitHub repository URL whereas previously reviewed tools provided full GitHub integration with capability of selection of repositories from a list view. The basic Codeship builds do not currently support YAML-based configurations, but this capability has been added to Docker-based containers (CodeJet). For basic build the only configurations needed were selection of the JDK version using jdk_switcher command and definition of standard Maven goals. Firefox browser tests worked without any addition configurations. In general everything worked as expected, but the service would benefit from YAML based configurations. Homepage: https://app.shippable.com Platforms: Linux VCS support: GitHub, Bitbucket Pipeline configuration: No (Currently YAML support only for Docker-based CodeJet) Pricing: Free for 1 concurrent build and 1 parallel pipeline (1 build container, 100 builds, 5 projects, unlimited teams) CodeJet with Docker containers starting from $75/month On-Site: No Maturity: Founded 2011 Over 75k user accounts References: Sears, CareerBuilder, BlastWorks,… API: REST Browser testing: Firefox, Chrome, PhantomJs, CasperJs, SlimerJs Other: SSH access for debugging builds. SolanoCI: The advantage of SolanoCI is it’s wide support for different repositories and version control systems. Whereas all previously reviewed CI tools were limited to GitHub and/or Bitbucket repositories, SolanoCI provides GitHub, GitLab, BitBucket, Stash, Git and Mercurial support. The build configurations for SolanoCI are made using solano.yml configuration file and configuration was easy to setup. The user interface was not the easiest to use and there seemed to be minor bugs related to accessing build logs. The example solano.yml configuration is show below: java: java_version: java-8-oracle maven_version: '3.2.3' firefox: '27.0.1' tests: - mvn clean verify Homepage: https://www.solanolabs.com VCS support: Git, GitHub, BitBucket, Stash, GitLab, Mercurial Pipeline configuration: YAML Pricing: Starting at $15/month (2 workers, 10 worker hours) 14-day free trial available On-Site deployment: Yes. Solano Private CI Maturity: Founded 2011 Customer references: AirBnB, New Relic,… API: REST Browser testing support: Firefox, Chrome Other: SSH Debug Console (in Private release) Docker container support Android SDK And the long list of alternatives I limited my review to these six hosted CI platforms, but there is still at least dozen other popular alternatives providing similar features and capabilities, such as: Drone.io YAML configuration GitHub and BitBucket repositories AppVeyor Continuous delivery for Windows applications YAML configuration GitHub, BitBucket, Visual Studio Online, GitLab, Kiln, Subversion, Git, Mercurial and Stash repositories Wercker YAML configuration Container-centric platform for automation HostedCI Jenkins based hosted CI for iOS and OS X projects Full Git, Mercurial and SVN repository support SemaphoreCI YAML configuration Docker optimized CI platform Bitrise CI for mobile platforms (iOS, Android, Xamarin) YAML configuration Run locally with CLI Summary Nowadays there is available dozens of alternative hosted CI platforms. Most of there are capable of providing support for complex build pipeline configurations, parallelization and integrations support with 3rd party services. As the pricing and features vary, the selection of the platform should be made based on the requirements set for the project. Based on this short review, I personally liked usability of the TravisCI and CircleCI. Both of these provided very similar approach for configuring and managing build pipelines using YAML configuration files. In addition, the SnapCI provided very clean and intuitive user interface to configure builds via web-based user interface.WebDriver locator performance - Is Edge really that fast?2016-04-11T21:03:00+03:002020-03-26T17:09:00+02:00https://jperala.fi/2016/04/11/webdriver-locator-performance<p>With colleagues we had discussion about the performance of different WebDriver locators. According some sources especially the XPATH locator can have slow performance compared to CSS and other locators. To get rough idea about the performance of different locators I decided to make a quick test on Firefox, Chrome and Edge.</p>
<h2 id="test-case">Test case</h2>
<p>Test measures time used to find an element from <a href="http://twitter.com">Twitter</a> front page using different selectors. The finding of element is repeated 100 times to get more reproducible and statistically reliable results. The locators used in test were:</p>
<ul>
<li>By.name(“session[username_or_email]”)</li>
<li>By.className(“email-input”)</li>
<li>By.id(“signin-email”)</li>
<li>By.linkText(“Forgot password?”)</li>
<li>By.xpath(“//*[@id="signin-email"]”)</li>
<li>By.cssSelector(“input.text-input.email-input”)</li>
</ul>
<h2 id="test-environment">Test Environment</h2>
<ul>
<li>Windows 10
<ul>
<li>Firefox 45.0.1</li>
<li>Chrome 49.0.2623.110</li>
<li>Edge 25.10586.0.0</li>
<li>Java 8 (64bit, version 1.8.0_31)</li>
</ul>
</li>
<li>WebDriver 2.53.0
<ul>
<li>FirefoxDriver 2.53.0</li>
<li>ChromeDriver 2.21.371459</li>
<li>MicrosoftWebDriver 3.14316</li>
</ul>
</li>
</ul>
<h2 id="results">Results</h2>
<p>Here are the measurement results. The time is total time of 100 findElement() method calls using the given locator. The slowests execution times per each browser are <strong>bolded</strong>.</p>
<table>
<tbody>
<tr>
<td><strong>Locator</strong></td>
<td><strong>Firefox</strong></td>
<td><strong>Chrome</strong></td>
<td><strong>Edge</strong></td>
</tr>
<tr>
<td><strong>name</strong></td>
<td>1377 ms</td>
<td>929 ms</td>
<td>204 ms</td>
</tr>
<tr>
<td><strong>className</strong></td>
<td>1795 ms</td>
<td>902 ms</td>
<td>199 ms</td>
</tr>
<tr>
<td><strong>id</strong></td>
<td>1600 ms</td>
<td>851 ms</td>
<td><strong>262 ms</strong></td>
</tr>
<tr>
<td><strong>linkText</strong></td>
<td><strong>9056 ms</strong></td>
<td><strong>1522 ms</strong></td>
<td>238 ms</td>
</tr>
<tr>
<td><strong>xpath</strong></td>
<td>2229 ms</td>
<td>919 ms</td>
<td>247 ms</td>
</tr>
<tr>
<td><strong>cssSelector</strong></td>
<td>1280 ms</td>
<td>809 ms</td>
<td>219 ms</td>
</tr>
</tbody>
</table>
<h2 id="summary">Summary</h2>
<p>The executed test is just a quick measurement to get rough idea about the locator performance. The performance is naturally affected by context (e.g. page content, locator queries and test environment), and therefore the results should not be used as conclusive results but rather as starting point for additional analysis.</p>
<p>However, what can be seen from the results is that the performance is highly affected by the used browser. In fact, the used browser seems to be more dominant factor for the performance than the used locator type. Supprisingly the Edge seems to beat Firefox and Chrome in performance by far. Can Edge really be this fast?</p>
<p>Another finding was very poor performance of linkText -locator in Firefox, whereas is Chrome and Firefox no similar performance drop was observable. Also bit supprisingly, the id -locator was not always the fastest locator (as we initially expected).</p>
<p>As conclusion, the performance of locators in most cases has only minimal (or none) effect on the test execution time and result varies on each browser. In most normal test cases the test execution times are affected on ten- to hundredfolds by the wait strategies used in tests.</p>
<h2 id="test-code">Test code</h2>
<p>If interested to carry out your own additional measurements, below is the test case used for the measurements.</p>
<figure class="highlight"><pre><code class="language-java" data-lang="java"><span class="kn">import</span> <span class="nn">org.junit.After</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.junit.Before</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.junit.Test</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.junit.runner.RunWith</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.junit.runners.Parameterized.Parameters</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.junit.runners.Parameterized</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.openqa.selenium.By</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.openqa.selenium.WebDriver</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.openqa.selenium.chrome.ChromeDriver</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.openqa.selenium.edge.EdgeDriver</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.openqa.selenium.firefox.FirefoxDriver</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.time.format.TextStyle</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.ArrayList</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.Arrays</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.Collection</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.List</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.concurrent.TimeUnit</span><span class="o">;</span>
<span class="nd">@RunWith</span><span class="o">(</span><span class="n">Parameterized</span><span class="o">.</span><span class="na">class</span><span class="o">)</span>
<span class="kd">public</span> <span class="kd">class</span> <span class="nc">WebDriverTest</span> <span class="o">{</span>
<span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">String</span><span class="o">[]</span> <span class="n">BROWSERS</span> <span class="o">=</span> <span class="o">{</span><span class="s">"firefox"</span><span class="o">,</span> <span class="s">"chrome"</span><span class="o">,</span> <span class="s">"edge"</span><span class="o">};</span>
<span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="kt">int</span> <span class="n">TEST_COUNT</span> <span class="o">=</span> <span class="mi">100</span><span class="o">;</span>
<span class="n">WebDriver</span> <span class="n">driver</span><span class="o">;</span>
<span class="n">By</span> <span class="n">by</span><span class="o">;</span>
<span class="n">String</span> <span class="n">locator</span><span class="o">;</span>
<span class="n">String</span> <span class="n">browser</span><span class="o">;</span>
<span class="nd">@Parameters</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="n">Collection</span> <span class="nf">locators</span><span class="o">()</span> <span class="o">{</span>
<span class="n">List</span><span class="o"><</span><span class="n">Object</span><span class="o">[]></span> <span class="n">params</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ArrayList</span><span class="o"><>();</span>
<span class="k">for</span> <span class="o">(</span><span class="n">String</span> <span class="n">browser</span> <span class="o">:</span> <span class="n">BROWSERS</span><span class="o">)</span> <span class="o">{</span>
<span class="n">params</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="k">new</span> <span class="n">Object</span><span class="o">[]{</span><span class="n">browser</span><span class="o">,</span> <span class="n">By</span><span class="o">.</span><span class="na">name</span><span class="o">(</span><span class="s">"session[username_or_email]"</span><span class="o">),</span> <span class="s">"name"</span><span class="o">});</span>
<span class="n">params</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="k">new</span> <span class="n">Object</span><span class="o">[]{</span><span class="n">browser</span><span class="o">,</span> <span class="n">By</span><span class="o">.</span><span class="na">className</span><span class="o">(</span><span class="s">"email-input"</span><span class="o">),</span> <span class="s">"className"</span><span class="o">});</span>
<span class="n">params</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="k">new</span> <span class="n">Object</span><span class="o">[]{</span><span class="n">browser</span><span class="o">,</span> <span class="n">By</span><span class="o">.</span><span class="na">id</span><span class="o">(</span><span class="s">"signin-email"</span><span class="o">),</span> <span class="s">"id"</span><span class="o">});</span>
<span class="n">params</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="k">new</span> <span class="n">Object</span><span class="o">[]{</span><span class="n">browser</span><span class="o">,</span> <span class="n">By</span><span class="o">.</span><span class="na">linkText</span><span class="o">(</span><span class="s">"Forgot password?"</span><span class="o">),</span> <span class="s">"linkText"</span><span class="o">});</span>
<span class="n">params</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="k">new</span> <span class="n">Object</span><span class="o">[]{</span><span class="n">browser</span><span class="o">,</span> <span class="n">By</span><span class="o">.</span><span class="na">xpath</span><span class="o">(</span><span class="s">"//*[@id=\"signin-email\"]"</span><span class="o">),</span> <span class="s">"xpath"</span><span class="o">});</span>
<span class="n">params</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="k">new</span> <span class="n">Object</span><span class="o">[]{</span><span class="n">browser</span><span class="o">,</span> <span class="n">By</span><span class="o">.</span><span class="na">cssSelector</span><span class="o">(</span><span class="s">"input.text-input.email-input"</span><span class="o">),</span> <span class="s">"cssSelector"</span><span class="o">});</span>
<span class="o">}</span>
<span class="k">return</span> <span class="n">params</span><span class="o">;</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="nf">WebDriverTest</span><span class="o">(</span><span class="n">String</span> <span class="n">browser</span><span class="o">,</span> <span class="n">By</span> <span class="n">by</span><span class="o">,</span> <span class="n">String</span> <span class="n">locator</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">browser</span> <span class="o">=</span> <span class="n">browser</span><span class="o">;</span>
<span class="k">this</span><span class="o">.</span><span class="na">by</span> <span class="o">=</span> <span class="n">by</span><span class="o">;</span>
<span class="k">this</span><span class="o">.</span><span class="na">locator</span> <span class="o">=</span> <span class="n">locator</span><span class="o">;</span>
<span class="o">}</span>
<span class="nd">@Before</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">before</span><span class="o">()</span> <span class="o">{</span>
<span class="k">switch</span> <span class="o">(</span><span class="n">browser</span><span class="o">)</span> <span class="o">{</span>
<span class="k">case</span> <span class="s">"firefox"</span><span class="o">:</span>
<span class="n">driver</span> <span class="o">=</span> <span class="k">new</span> <span class="n">FirefoxDriver</span><span class="o">();</span>
<span class="k">break</span><span class="o">;</span>
<span class="k">case</span> <span class="s">"chrome"</span><span class="o">:</span>
<span class="n">driver</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ChromeDriver</span><span class="o">();</span>
<span class="k">break</span><span class="o">;</span>
<span class="k">case</span> <span class="s">"edge"</span><span class="o">:</span>
<span class="n">driver</span> <span class="o">=</span> <span class="k">new</span> <span class="n">EdgeDriver</span><span class="o">();</span>
<span class="k">break</span><span class="o">;</span>
<span class="k">default</span><span class="o">:</span>
<span class="k">break</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="nd">@After</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">afterClass</span><span class="o">()</span> <span class="o">{</span>
<span class="n">driver</span><span class="o">.</span><span class="na">close</span><span class="o">();</span>
<span class="n">driver</span><span class="o">.</span><span class="na">quit</span><span class="o">();</span>
<span class="o">}</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">locatorPerformance</span><span class="o">()</span> <span class="kd">throws</span> <span class="n">InterruptedException</span> <span class="o">{</span>
<span class="c1">// load page
</span>
<span class="n">driver</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="s">"http://twitter.com"</span><span class="o">);</span>
<span class="c1">// dry-run and let browser settle a bit
</span>
<span class="n">driver</span><span class="o">.</span><span class="na">findElement</span><span class="o">(</span><span class="n">by</span><span class="o">);</span>
<span class="n">TimeUnit</span><span class="o">.</span><span class="na">SECONDS</span><span class="o">.</span><span class="na">sleep</span><span class="o">(</span><span class="mi">5</span><span class="o">);</span>
<span class="c1">// do measurements
</span>
<span class="kt">long</span> <span class="n">start</span> <span class="o">=</span> <span class="n">System</span><span class="o">.</span><span class="na">currentTimeMillis</span><span class="o">();</span>
<span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span> <span class="n">i</span> <span class="o"><</span> <span class="n">TEST_COUNT</span><span class="o">;</span> <span class="n">i</span><span class="o">++)</span> <span class="o">{</span>
<span class="n">driver</span><span class="o">.</span><span class="na">findElement</span><span class="o">(</span><span class="n">by</span><span class="o">);</span>
<span class="o">}</span>
<span class="kt">long</span> <span class="n">stop</span> <span class="o">=</span> <span class="n">System</span><span class="o">.</span><span class="na">currentTimeMillis</span><span class="o">();</span>
<span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">browser</span> <span class="o">+</span> <span class="s">": Locator "</span> <span class="o">+</span> <span class="n">locator</span> <span class="o">+</span> <span class="s">" execution time: "</span> <span class="o">+</span> <span class="o">(</span><span class="n">stop</span> <span class="o">-</span> <span class="n">start</span><span class="o">));</span>
<span class="o">}</span>
<span class="o">}</span></code></pre></figure>jperalaWith colleagues we had discussion about the performance of different WebDriver locators. According some sources especially the XPATH locator can have slow performance compared to CSS and other locators. To get rough idea about the performance of different locators I decided to make a quick test on Firefox, Chrome and Edge. Test case Test measures time used to find an element from Twitter front page using different selectors. The finding of element is repeated 100 times to get more reproducible and statistically reliable results. The locators used in test were: By.name(“session[username_or_email]”) By.className(“email-input”) By.id(“signin-email”) By.linkText(“Forgot password?”) By.xpath(“//*[@id="signin-email"]”) By.cssSelector(“input.text-input.email-input”) Test Environment Windows 10 Firefox 45.0.1 Chrome 49.0.2623.110 Edge 25.10586.0.0 Java 8 (64bit, version 1.8.0_31) WebDriver 2.53.0 FirefoxDriver 2.53.0 ChromeDriver 2.21.371459 MicrosoftWebDriver 3.14316 Results Here are the measurement results. The time is total time of 100 findElement() method calls using the given locator. The slowests execution times per each browser are bolded. Locator Firefox Chrome Edge name 1377 ms 929 ms 204 ms className 1795 ms 902 ms 199 ms id 1600 ms 851 ms 262 ms linkText 9056 ms 1522 ms 238 ms xpath 2229 ms 919 ms 247 ms cssSelector 1280 ms 809 ms 219 ms Summary The executed test is just a quick measurement to get rough idea about the locator performance. The performance is naturally affected by context (e.g. page content, locator queries and test environment), and therefore the results should not be used as conclusive results but rather as starting point for additional analysis. However, what can be seen from the results is that the performance is highly affected by the used browser. In fact, the used browser seems to be more dominant factor for the performance than the used locator type. Supprisingly the Edge seems to beat Firefox and Chrome in performance by far. Can Edge really be this fast? Another finding was very poor performance of linkText -locator in Firefox, whereas is Chrome and Firefox no similar performance drop was observable. Also bit supprisingly, the id -locator was not always the fastest locator (as we initially expected). As conclusion, the performance of locators in most cases has only minimal (or none) effect on the test execution time and result varies on each browser. In most normal test cases the test execution times are affected on ten- to hundredfolds by the wait strategies used in tests. Test code If interested to carry out your own additional measurements, below is the test case used for the measurements. import org.junit.After; import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; import org.junit.runners.Parameterized.Parameters; import org.junit.runners.Parameterized; import org.openqa.selenium.By; import org.openqa.selenium.WebDriver; import org.openqa.selenium.chrome.ChromeDriver; import org.openqa.selenium.edge.EdgeDriver; import org.openqa.selenium.firefox.FirefoxDriver; import java.time.format.TextStyle; import java.util.ArrayList; import java.util.Arrays; import java.util.Collection; import java.util.List; import java.util.concurrent.TimeUnit; @RunWith(Parameterized.class) public class WebDriverTest { private static final String[] BROWSERS = {"firefox", "chrome", "edge"}; private static final int TEST_COUNT = 100; WebDriver driver; By by; String locator; String browser; @Parameters public static Collection locators() { List<Object[]> params = new ArrayList<>(); for (String browser : BROWSERS) { params.add(new Object[]{browser, By.name("session[username_or_email]"), "name"}); params.add(new Object[]{browser, By.className("email-input"), "className"}); params.add(new Object[]{browser, By.id("signin-email"), "id"}); params.add(new Object[]{browser, By.linkText("Forgot password?"), "linkText"}); params.add(new Object[]{browser, By.xpath("//*[@id=\"signin-email\"]"), "xpath"}); params.add(new Object[]{browser, By.cssSelector("input.text-input.email-input"), "cssSelector"}); } return params; } public WebDriverTest(String browser, By by, String locator) { this.browser = browser; this.by = by; this.locator = locator; } @Before public void before() { switch (browser) { case "firefox": driver = new FirefoxDriver(); break; case "chrome": driver = new ChromeDriver(); break; case "edge": driver = new EdgeDriver(); break; default: break; } } @After public void afterClass() { driver.close(); driver.quit(); } @Test public void locatorPerformance() throws InterruptedException { // load page driver.get("http://twitter.com"); // dry-run and let browser settle a bit driver.findElement(by); TimeUnit.SECONDS.sleep(5); // do measurements long start = System.currentTimeMillis(); for (int i = 0; i < TEST_COUNT; i++) { driver.findElement(by); } long stop = System.currentTimeMillis(); System.out.println(browser + ": Locator " + locator + " execution time: " + (stop - start)); } }Going static - From WordPress to Jekyll2016-02-28T00:25:00+02:002020-03-26T17:08:00+02:00https://jperala.fi/2016/02/28/going-static<p>I am a lazy blogger - I only write occasionally. Originally I setup my blog to get familiar with Content Management Systems (CMSs). For the last 2 years this blog has been hosted on cheap $5/month VPS server running WordPress. Although I write only few posts per year, maintaining the server requires some additional effort on monthly basis. The server and CMS needs to be kept up-to-date to avoid security risks, backups needs to be managed, and so on. Especially the security concerns have been topic during the last year as numerous security vulnerabilities have been identified in the WordPress. There is also the cost issue, although insignificant, caused by the maintenance of the VPS server hosting. So it is time to try something new.</p>
<p>The easiest alternative would have been to move the blog to any hosted blogging platform such <em>Wordpress.com</em> or <em>Blogger.com</em>. However, I am a tech guy by nature and like to know what happens under the hood of the system. Therefore I started looking alternative blogging platform that would require less maintenance than WordPress, but yet give full control over the system.</p>
<p>Static web-page generators like <a href="http://jekyllrb.com/">Jekyll</a> and <a href="https://gohugo.io/">Hugo</a> have gained huge popularity during last years. With static web-page generators you write your posts and content as plain text files with markup syntax, which is then compiled into static HTML website using the template files. Although the static sites are somewhat more limited than full-blown CMS systems, there is also notable benefits:</p>
<ul>
<li>Simplicity.
<ul>
<li>No databases or server-side processing, only static files deployed on file-server.</li>
<li>No more database backups. Just keep your site source files in version control and you’re backed up.</li>
</ul>
</li>
<li>Security.
<ul>
<li>No more security risks caused by the CMS platform or plugin vulnerabilities.</li>
<li>No maintenance of CMS and plugins versions, users and roles.</li>
</ul>
</li>
<li>Cost
<ul>
<li>Site can be deployed on low-cost file-server (e.g., <a href="https://aws.amazon.com/s3">Amazon S3</a> or freely on <a href="https://github.io">github.io</a>).</li>
<li>No need for VPS server or Amazon EC2 resources.</li>
</ul>
</li>
</ul>
<p>After going through the benefits, I decided to migrate my blog as static <em>Jekyll</em> website. Jekyll is used to power the GitHub pages, so the tool is well matured and has large user base. Also, the Jekyll site can be freely hosted on GitHub pages, so there is no longer need to maintain separate VPS for blog hosting.</p>
<h3 id="from-wordpress-to-jekyll">From Wordpress to Jekyll</h3>
<p>Migrating from Wordpress to Jekyll was surprisingly straightforward.</p>
<h3 id="install-jekyll">Install Jekyll</h3>
<p>Installing Jekyll with gem (detailed instructions in <a href="http://jekyllrb.com/">Jekyll</a>):</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">gem <span class="nb">install </span>jekyll</code></pre></figure>
<h3 id="create-new-project-template-or-fork-a-jekyll-theme">Create new project template (or fork a Jekyll theme)</h3>
<p>Create new jekyll site and deploy it locally:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">jekyll new my-website
<span class="nb">cd </span>my-website
jekyll serve</code></pre></figure>
<p>Alternatively, clone a Jekyll theme you want to use, e.g.:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">git clone https://github.com/BlackrockDigital/startbootstrap-clean-blog-jekyll.git my-website
<span class="nb">cd </span>my-website
jekyll serve</code></pre></figure>
<p>Once the jekyll is serving the site locally, the site can be previewed on http://localhost:4000.</p>
<h3 id="export-existing-posts-from-wordpress-to-jekyll">Export existing posts from Wordpress to Jekyll</h3>
<p>Go to WordPress site and export the site as <em>wordpress.xml</em> file. Once exported, run the jekyll-import that generates the posts into Jekyll project:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">gem <span class="nb">install </span>jekyll-import
ruby <span class="nt">-rubygems</span> <span class="nt">-e</span> <span class="s1">'require "jekyll-import";
JekyllImport::Importers::WordpressDotCom.run({
"source" => "wordpress.xml",
"no_fetch_images" => false,
"assets_folder" => "assets"
})'</span></code></pre></figure>
<p><em>Note:</em> You may need to go through the generated posts in the <em>_posts/</em> folder to fix invalid links and image references to old Wordpress site.</p>
<h3 id="deploy-site">Deploy site</h3>
<p>If you want to deploy the Jekyll site to free GitHub pages, commit the project to your github.com account. Just name the repository as <em><yourname>.github.io</em> and the github automatically builds the site into the <em>http://<yourname>.github.io</em>.</p>
<p>Alternatively, if you want to deploy the site to another file-server, build the site locally and deploy generated static site from <em>_site/</em> folder to fileserver of your choice:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">jekyll build
<span class="nb">cd </span>_site</code></pre></figure>jperalaI am a lazy blogger - I only write occasionally. Originally I setup my blog to get familiar with Content Management Systems (CMSs). For the last 2 years this blog has been hosted on cheap $5/month VPS server running WordPress. Although I write only few posts per year, maintaining the server requires some additional effort on monthly basis. The server and CMS needs to be kept up-to-date to avoid security risks, backups needs to be managed, and so on. Especially the security concerns have been topic during the last year as numerous security vulnerabilities have been identified in the WordPress. There is also the cost issue, although insignificant, caused by the maintenance of the VPS server hosting. So it is time to try something new. The easiest alternative would have been to move the blog to any hosted blogging platform such Wordpress.com or Blogger.com. However, I am a tech guy by nature and like to know what happens under the hood of the system. Therefore I started looking alternative blogging platform that would require less maintenance than WordPress, but yet give full control over the system. Static web-page generators like Jekyll and Hugo have gained huge popularity during last years. With static web-page generators you write your posts and content as plain text files with markup syntax, which is then compiled into static HTML website using the template files. Although the static sites are somewhat more limited than full-blown CMS systems, there is also notable benefits: Simplicity. No databases or server-side processing, only static files deployed on file-server. No more database backups. Just keep your site source files in version control and you’re backed up. Security. No more security risks caused by the CMS platform or plugin vulnerabilities. No maintenance of CMS and plugins versions, users and roles. Cost Site can be deployed on low-cost file-server (e.g., Amazon S3 or freely on github.io). No need for VPS server or Amazon EC2 resources. After going through the benefits, I decided to migrate my blog as static Jekyll website. Jekyll is used to power the GitHub pages, so the tool is well matured and has large user base. Also, the Jekyll site can be freely hosted on GitHub pages, so there is no longer need to maintain separate VPS for blog hosting. From Wordpress to Jekyll Migrating from Wordpress to Jekyll was surprisingly straightforward. Install Jekyll Installing Jekyll with gem (detailed instructions in Jekyll): gem install jekyll Create new project template (or fork a Jekyll theme) Create new jekyll site and deploy it locally: jekyll new my-website cd my-website jekyll serve Alternatively, clone a Jekyll theme you want to use, e.g.: git clone https://github.com/BlackrockDigital/startbootstrap-clean-blog-jekyll.git my-website cd my-website jekyll serve Once the jekyll is serving the site locally, the site can be previewed on http://localhost:4000. Export existing posts from Wordpress to Jekyll Go to WordPress site and export the site as wordpress.xml file. Once exported, run the jekyll-import that generates the posts into Jekyll project: gem install jekyll-import ruby -rubygems -e 'require "jekyll-import"; JekyllImport::Importers::WordpressDotCom.run({ "source" => "wordpress.xml", "no_fetch_images" => false, "assets_folder" => "assets" })' Note: You may need to go through the generated posts in the _posts/ folder to fix invalid links and image references to old Wordpress site. Deploy site If you want to deploy the Jekyll site to free GitHub pages, commit the project to your github.com account. Just name the repository as <yourname>.github.io and the github automatically builds the site into the http://<yourname>.github.io. Alternatively, if you want to deploy the site to another file-server, build the site locally and deploy generated static site from _site/ folder to fileserver of your choice: jekyll build cd _site