Monday, July 29, 2019

Python - Keywords


Python Keywords
Keywords are the reserved words and we cannot use a keyword as variable name, function name or an Identifier. Almost 33 keywords are available in Python which is quite lesser than other languages like Java, C++ etc.

Method
Description
and
A logical operator
as
To create an alias
assert
For debugging
break
To break out of a loop
class
To define a class
continue
To continue to the next iteration of a loop
def
To define a function
del
To delete an object
elif
Used in conditional statements, same as else if
else
Used in conditional statements
except
Used with exceptions, what to do when an exception occurs
False
Boolean value, result of comparison operations
finally
Used with exceptions, a block of code that will be executed no matter if there is an exception or not
for
To create a for loop
from
To import specific parts of a module
global
To declare a global variable
if
To make a conditional statement
import
To import a module
To check if a value is present in a list, tuple, etc.
To test if two variables are equal
lambda
To create an anonymous function
None
Represents a null value
nonlocal
To declare a non-local variable
A logical operator
or
A logical operator
pass
A null statement, a statement that will do nothing
raise
To raise an exception
return
To exit a function and return a value
True
Boolean value, result of comparison operations
try
To make a try...except statement
while
To create a while loop
with
Used to simplify exception handling
yield
To end a function, returns a generator



                                                  List of Key Words

There are some basic rules that need to be followed with the keywords.No identifiers should have the same name as any of the keywords.Names that begin with __ underscore should not be used.







Saturday, July 20, 2019

Python - Datatype


Every thing in python has a datatype. There are various types of datatypes in python. Broadly they are divided into numbers and strings. Let's have a look at both.

Python Numbers

Integers, float and complex numbers fall under python’s number category. They are represented in python in following ways:

Int

Integers can be of any length, it is only limited by the memory available.

Float

A floating point number is accurate up to 15 decimal places. Integer and floating points are separated by decimal points.

Complex

Complex numbers are written in the form, x + yj, where x is the real part and y is the imaginary part.

How would you know a data type of a variable?


type() function to know which class a variable or a value belongs.

type at work

Python Strings



String is sequence of unicode characters. We can use single quotes or double quotes to represent strings. Multiline strings can be denoted using triple quotes''' or """.

Conversion of data type

We can convert between different data types by using different type conversion functions like int(), float(), str() etc.



Python - Introduction

As per 2018 survey conducted by Stackoverflow.com , Python is the 7th most sought language and most wanted technology.The question is why python is so demanding ? Here are the few points that i have observed and gathered through books and internet.

Python is hot

According to research by Dice Python is also one of the hottest skills to have and the most popular programming language in the world based on the Popularity of Programming Language Index.

Python is interpreted

Most of the languages that are in use are compiled one , it means the language are first converted into machine code , the language of our processors before it can run.
However, python is an interpreted language and the codes are not converted
into machine code.

Python is free

The Python interpreter is developed under an OSI-approved open-source license, making it free to install, use, and distribute, even for commercial purposes.

Python is portable

Python code is interpreted and not compiled into native machine instructions, code written for one platform will work on any other platform that has the Python interpreter installed.

Python is simple

Python 3 has 33 keywords, and Python 2 has 31. By contrast, C++ has 62, Java has 53, and Visual Basic has more than 120, though these latter examples probably vary somewhat by implementation or dialect.

Friday, July 19, 2019

Python - Match vs Search


Regular Expression are very useful and can be used by developers in numerous of ways. It proved handy when we are encountering with the data having some sort of patterns. There are numerous of operation that can be done using regular expression. The re module can be used to import the regular expression module.

Match Operation in Python :-   Python match operation will look for the first matching string in a given set of data. If the searched data is not available at the first position, then it will return NONE.

Example :-  We will check what will be the output when we look for the data set that is available at the first place and when it is not at the first index position.



Python Code :- (when at the first position)
import re
data = "this is good"
res_match = re.match("this",data)
print("The result set when it is at first place",res_match)

Output :-
The result set when it is at first place <re.Match object; span=(0, 4), match='this'>

Python Code :- (when not at first position)
import re
data = "this is good"
res_mat_pos = re.match("good",data)
print("The result set when it is not in first place",res_mat_pos)
Output :- 
The result set when it is not in first place None.

Search Operation in Python :- Search operation works in the similar way as match works , except that it can search the word that are in different index position.

Sample Python Code :-
import re
data = "this is good"
result = re.search("good",data)
print("This is the result for the search",result)
Output :-
This is the result for the search <re.Match object; span=(2, 4), match='is'>

Thanks !!!

Thursday, June 27, 2019

Sqoop - Eval


Eval is normally used to run simple sql queries against the database server.It will also preview the result on the console level.Eval can be used to evaluate any type of query like DDL/DML.

EVAL is only to check the database connection and also to preview the small set of data.It will provide the user to test the simple queries.

Sqoop Syntax :-

sqoop --eval {generic-args} {eval args}

Generic args :-

Argument Description
connect <jdbc-uri>
Specify JDBC connect string
connection-manager <class-name>
Specify connection manager class to use
driver <class-name>
Manually specify JDBC driver class to use
hadoop-mapred-home <dir>
Override $HADOOP_MAPRED_HOME
help
Print usage instructions
password-file
Set path for a file containing the authentication password
-P
Read password from console
password <password>
Set authentication password
username <username>
Set authentication username
verbose
Print more information while working
connection-param-file <filename>
Optional properties file that provides connection parameters
relaxed-isolation
Set connection transaction isolation to read uncommitted for the mappers.


Eval Args :-

e ,query Execute statement in SQL.

query examples :-

sqoop eval \
–connect jdbc:mysql://localhost/test \
–username root \
–query “SELECT * FROM sqoop_test ”





Monday, June 17, 2019

Hive - Reading JSON using Serde

Reading JSON in Hive is quite trickier but JSON serde has made our life easier.The JSON serde can be downloaded from any third party website.The JSON serde for JSON file is available in Hive 0.12 and later version.

We can download JSON serde from any of the sites like :-

https://code.google.com/archive/p/hive-json-serde/downloads

We have downloaded a sample JSON file  from https://support.oneskyapp.com/hc/en-us/articles/208047697-JSON-sample-files


Step 1.) Download the Serde from the above link and store it in any of the required folder .



Step 2.) Once the JSON serde has been downloaded , kindly add it to the Hive using below command.


ADD path/to/jar Jarfile.


Step 3.) Create the required table format using the below syntax .

create external table serde_fruit(fruit string,size string,color string) ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.JsonSerde' location ‘/serde_ex’;



Step 4. ) validate the data by using the select query.




Delta Lake - Time Travel

  Time Travel allows you to query, restore, or compare data from a previous version of a Delta table. Delta Lake automatically keeps tra...