Selenium Python Tutorial 1 | Selenium WebDriver python | Python Automation Testing | Python WebDriver

This is the first tutorial in the Selenium Python Tutorials for Beginners. This Selenium Python beginner tutorial gives an introduction to Python and Selenium WebDriver, pip install Selenium and multiple selenium python example. First, view the Selenium Python tutorial 1. Then read below.

You can do Python browser automation with Selenium WebDriver. Python programming language is also used for web development, game development, machine learning and other applications. The Selenium library Python is open source and free. Selenium tests web applications in multiple browsers.

In order to work with selenium using python, we need to install Python 3, which we can install from the Python official website. Next, how to Python install Selenium? We can use pip, which is the Python package installer. In the command prompt, the pip install Selenium command is below. This command gets the Selenium Python download or the Selenium package from the Selenium official website.

pip install -U selenium

For Python Selenium automation, we can create a new project in the PyCharm IDE. Click File> New Project. Then give the name of the project. In order to configure Selenium WebDriver in this project, select the project. Click File > Settings > Project > Project interpreter. It shows the packages that are available. Click + button, which is the Install button. It shows all the available packages. In the search text box, type selenium. Select selenium and click Install Package button. It gives the message, Package 'selenium' installed successfully. Click close button. Then it shows selenium in the packages list also. Click OK button. Next, right-click the Project > New > Python file. Give a name and click OK button. We need to download the Python WebDriver libraries for the browsers. We can go to the Selenium website  and click on the Downloads tab. In the Selenium Client & WebDriver Language Bindings section, find Python and click the Download link. Here are the drivers for the browsers like Firefox (GeckoDriver), Chrome (ChromeDriver) and Edge (EdgeDriver) which we can download. Now, we are ready for selenium using python. Here is my first Selenium Python example:

# Selenium WebDriver Python coding
# Import WebDriver library from the Selenium package
from selenium import webdriver
# Import By to find elements on the web page
from selenium.webdriver.common.by import By

# Give the location of the browser driver. r means a raw Python string.
exec_path = r"E:\Training\SeleniumPython\Downloads\geckodriver-v0.26.0-win32\geckodriver.exe"
URL = r"https://www.wikipedia.org/"
english_link_locator = "js-link-box-en"
search_locator = "searchInput"
search_text = "Software"

# Define the Selenium WebDriver variable with the executable path.
driver = webdriver.Firefox(executable_path=exec_path)
# Navigate to the URL.
driver.get(URL)
driver.maximize_window()
# Find the English link.
english_link_element = driver.find_element(By.ID, english_link_locator)
english_link_element.click()
# Find the Search text box.
input_box_element = driver.find_element(By.ID, search_locator)
input_box_element.send_keys(search_text)
input_box_element.submit()
# The quit method closes the browser windows.
# driver.quit()

Here is my second Selenium Python example with WebDriverWait Python using multiple language link locators :

# Selenium WebDriver Python coding
from selenium import webdriver
from selenium.webdriver.common.by import By
# Import statements for explicit wait
from selenium.webdriver.support.ui import WebDriverWait as W
from selenium.webdriver.support import expected_conditions as E
import time

exec_path = r"E:\Training\SeleniumPython\Downloads\geckodriver-v0.26.0-win32\geckodriver.exe"
URL = r"https://www.wikipedia.org/"
# Define a list of locators for the language links.
language_locators = ["js-link-box-en", "js-link-box-ru", "js-link-box-de"]
search_locator = "searchInput"
search_text = "Software"
wait_time = 5

driver = webdriver.Firefox(executable_path=exec_path)
driver.get(URL)
# Define the wait variable for explicit wait.
wait = W(driver, wait_time)
driver.maximize_window()
for i in range(len(language_locators)):
    language_link = wait.until(E.presence_of_element_located((By.ID, language_locators[i])))
    language_link.click()
    input_box_element = wait.until(E.presence_of_element_located((By.ID, search_locator)))
    input_box_element.send_keys(search_text)
    input_box_element.submit()
    # Pause the script for a few seconds.
    time.sleep(4)
    driver.back()
    driver.back()
# driver.quit()

That is all in this Python Automation Testing tutorial. Want to learn more details like how to find the locators? Want to know about the synchronization issue in Test Automation and how to resolve it? Or want to see the above Selenium Python code working? Please view my Selenium Python tutorial 1. Thank you.

Big Data Tutorial 3 | big data in Healthcare | Big data testing

This is the third tutorial in the Big Data Tutorials for Beginners. This Big Data beginner tutorial explains Big Data in HealthCare, Big Data challenges, Big Data Testing, Big Data Testing challenges and Big Data Testing Tools. Please view the Big Data tutorial 3 or read on... First, let us learn about big data in healthcare. The healthcare industry is highly regulated and uses healthcare big data like patient health records, laboratory test results and test reports, prescriptions, claims and payments.

The main problem has been to analyze the healthcare big data quickly. The Hadoop framework is widely used in the healthcare industry to host the big data for quick processing by Map Reduce jobs. The use cases that I mentioned in my Big Data Tutorial 2 are applicable in the healthcare industry e.g. 360-degree view creation of patients and physicians and patient classification for care personalization and efficiency.  Big data examples in healthcare may enable improved prescription accuracy, reduced treatment cost and epidemic prediction. In the future, big data will be used to provide continuous patient monitoring using wearable sensors and Internet of Things devices.

Big data challenges are to perform Data Capture, Data Storage and Data Transfer actions quickly and cost effectively and to blend the data in multiple formats together in Data Analysis. For example, one of the the challenges in Data Capture is data ingestion in Hadoop. Data ingestion means migrating data from source systems to a Hadoop cluster. Since there can be numerous source systems and different ways to ingest data to Hadoop, it can become very complex. Big Data Search, Data Sharing, Data Visualization and Information Privacy are also challenging.

Big Data Testing: Big data testing deals with testing the data quality. High quality big data allows an organization to take accurate business decisions. Big data testing includes big data applications testing, data testing, functional testing and performance testing. Data testing includes:

•  Data Staging Validation: It is data ingestion testing. It validates the data being loaded into the Hadoop framework. It compares the source data with the data loaded into Hadoop. It also tests that data has been correctly loaded into the Hadoop framework at the correct location. Data staging validation checks the completeness, accuracy, integrity, consistency, validity, standardization and lack of duplicates in the data. Data staging validation of structured data is simpler than that of semi-structured data and unstructured data.
• Map Reduce Validation: It is the data processing testing to test the business logic and the outputs of the big data applications working on Hadoop. Map reduce validation checks that the Map Reduce process implements the data segregation and data aggregation rules and generates the key value pairs correctly.
• Output Validation: This is the output testing to test that the Hadoop data matches with the data moved into target systems like data warehouses. Output validation checks the data quality of output data files generated by Hadoop. Then, it tests the ETL process. Finally, it compares the Hadoop data to check complete and accurate data load in the target system.

Functional testing of the big data applications consists of testing the functionality of the big data applications provided in their user interface. Performance testing of the big data applications consists of measuring the data ingestion speed and data processing speed (of Map Reduce jobs) with metrics like throughput (of data ingestion), core utilization and memory utilization. Performance testing includes failover testing (to find if Big Data processing continues in the presence of failed nodes) and sub-component performance testing (to test each component of the Hadoop framework in isolation).

Big Data Testing challenges include availability of enough source test data, QA environment complexity and needing skills to build it, unstructured data testing complexity and needing multiple tools and test automation of big data testing requiring high skills (because unforeseen issues that may occur in unstructured data).

Big Data Testing tools: the Big Data Tester can use the tools in the Hadoop ecosystem for big data testing. Due to the complexity of the big data QA environment and big data volume, velocity and variety, no single tool can do end to end big data testing currently. Some big data testing tools are

• Tricentis Tosca BI and Data Warehouse Testing tests data integrity with built-in automated tests like pre-screening tests, ETL tests like completeness, uniqueness and referential integrity tests and other tests. 
• QuerySurge compares the source and target data systems and highlights data differences automatically. It also has features like test management integration, test monitoring and reporting and it's own API.
• TestingWhiz works with Hadoop, MongoDB and Teradata. It allows data validations tests and performance tests in big data testing.

Want to learn more including Big Data challenges in Healthcare and Big Data Questions and Answers? Please view my Big Data tutorial 3. Thank you.

Big Data Tutorial 2 | big data Analytics | Hadoop tutorial for beginners

This is the second tutorial in the Big Data Tutorials for Beginners. It is also the Hadoop Tutorial for Beginners. This Big Data beginner tutorial explains what is data analytics, big data analytics, Big data and Hadoop, big data applications, big data analytics tools, big data visualization tools and big data use cases. Please view the Big Data tutorial 2 or read on... What is Data Analytics? It means the analyses of data sets to find patterns and insights. Data Analytics uses multiple technologies and techniques. Data Analytics enable informed business decisions.

As shown above, Data Analytics can be divided into the following sub-categories :

• Descriptive analytics: analysis of past data to describe the current state
• Predictive analytics: data analysis to find patterns and forecast the future situation
• Prescriptive analytics: data analysis to recommend actions to exploit an advantage or mitigate a future issue

Next, what is Big Data Analytics? It is the process to examine and analyze big data to find patterns, correlations and trends. Big Data Analytics allows data analysts to make informed decisions faster. Big Data Analytics includes techniques like natural language processing, statistics, machine learning, predictive analytics and data mining to draw inferences. Big Data Analytics tools include Hadoop and related tools like HBase, Hive and Pig.

Big Data and Hadoop:  Hadoop is a big data open source tool. It is an open-source framework created by the Apache Software Foundation. Apache Hadoop uses distributed storage (many computers) to handle big data. Hadoop uses the Map Reduce data analysis technique. Hadoop has two components 1) HDFS (Hadoop Distributed File System) manages the big data storage 2) MapReduce manages the data processing. Hadoop divides data into many blocks and distributes these blocks across the computers in a cluster. Then, Hadoop sends code to the nodes for data processing using the Map and Reduce technique. All the tools used by an organization in it's big data architecture form the big data stack. Some tools in the Hadoop ecosystem include:

• Apache Hadoop YARN is the resource manager, job scheduler and job monitor in Hadoop.
• Apache HBase is the distributed database that works on HDFS in big data. HBase is a non-relational database that stores data as key-value pairs.
• Apache Hive is a tool for the purpose of data querying and analysis. Hive allows SQL-like queries to fetch data from the HDFS and the databases managed by Hadoop.
•  Apache Mahout is a tool for machine learning and data mining tasks.
• Apache Pig is a platform to write code to run on Hadoop. Pig uses Pig Latin which makes it easier to write programs using the Map Reduce technique.
• Apache Ambari is a tool to provision, manage and monitor Hadoop clusters.
• Apache Spark is a compute engine for massive data. Spark big data offers a programming model for ETL, streaming, machine learning and graph generation. In order to use Apache Spark, we can write programs using Java, Python, Scala, R or SQL.

There are many big data applications or big data tools that help organizations create their custom applications. Some examples of big data tools are Teradata database (to import data to Hadoop, query data and export data from Hadoop) and big data analytics software like Statistica (for predictive analytics), IBM's Watson Analytics and MongoDB (for querying unstructured data). Big data analytics means analysis of big data sets to find patterns and extract insights. Some examples of big data analytics tools are Tableau Public, Knime, Plotly and ElasticSearch. Some of the popular big data visualization tools are Tableau, Google Chart and D3.js. There are other tools for big data visualization like DataWrapper, FusionCharts and Plotly.

Some of the popular big data use cases applicable to many industry domains are shown above. These are:

• 360-degree view creation of an entity (e.g the customer or the patient or the student)
• Customer classification (into several categories) for relevant communication
• Price optimization based on demand, competition and customer profiles (especially useful in eCommerce, airline and hotel industries)
• New product/ service development (based on features that contribute to success)
• Distribution optimization (based on forecasted demand, expected traffic conditions and so on)
• Fraud prevention (to flag potentially fraudulent transactions in real-time)

These are just a few big data analytics examples. Big data analytics enables risk assessment in the insurance industry, product recommendation in eCommerce and customer care in every industry.

Want to learn more details about Big Data tools? Also, see Big Data questions and answers? Please view my Big Data tutorial 2. Thank you.