Hive - Reading JSON using Serde

Reading JSON in Hive is quite trickier but JSON serde has made our life easier.The JSON serde can be downloaded from any third party website.The JSON serde for JSON file is available in Hive 0.12 and later version.

We can download JSON serde from any of the sites like :-

https://code.google.com/archive/p/hive-json-serde/downloads

We have downloaded a sample JSON file  from https://support.oneskyapp.com/hc/en-us/articles/208047697-JSON-sample-files


Step 1.) Download the Serde from the above link and store it in any of the required folder .



Step 2.) Once the JSON serde has been downloaded , kindly add it to the Hive using below command.

ADD path/to/jar Jarfile.


Step 3.) Create the required table format using the below syntax .

create external table serde_fruit(fruit string,size string,color string) ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.JsonSerde' location ‘/serde_ex’;



Step 4. ) validate the data by using the select query.

Python - PostgreSQL DB connection

Hello friends , in this blog post we will learn how to connect python with PostgreSQL.PostgreSQL is a powerful, open source object-relational database system that uses and extends the SQL language combined with many features that safely store and scale the most complicated data workloads.PostgreSQL has earned a strong reputation for its proven architecture, reliability, data integrity, robust feature set, extensibility, and the dedication of the open source community behind the software to consistently deliver performant and innovative solutions. PostgreSQL runs on all major operating systems, has been ACID-compliant since 2001.

Step 1 : ) Create a table in PostgreSQL , here we have created a table called test.

Step 2 :) import module psycopg2 module from python repository

Type ‘pip install psycopg2’ from command line terminal

Step 3:) The required code to connect python with PostgreSQL.

import psycopg2

def connect():

conn_string = "host='localhost' dbname='postgres' user='postgres' password='postgres'"

print ("Print the connection details \n ->%s" % (conn_string))

conn = psycopg2.connect(conn_string)

cursor = conn.cursor()

cursor.execute("SELECT * FROM test")

display_data=cursor.fetchall()

print(display_data)

cursor.close()

if __name__== '__main__':

connect()

Step 4:) We can check the output on the console.