Author: Priyanka Nemana
Introduction:
In this blog, we have a detailed description of how to implement the Natural language processing(NLP) model in a website.
Contents
1. Case Study
2. What is NLP?
3. What is a User Interface?
4. Implementation of NLP
Execution of NLP in Django (Backend Framework)
NLP results at the front end (HTML)
5. Demo
6. Conclusion
7. References
1. Case Study
In our project “Travelverse”, we have designed a Tourism website where Users can visit and search for a location where they want to Visit. At the backend, we process the input given by the user and display the recommendation of popular places at that location. We have included NLP in the website backend; through this, a user can enter any text saying “I WANT TO VISIT AGRA”, and this text is passed as a parameter to the python code in Django (Backend Framework) which is, in turn connected to snowflake with python-snowflake connector module. Then, the locationtagger library of the NLP toolkit (NLTK) will extract the city name from the text ,which is AGRA, and then by executing a snowflake query, we can retrieve the details that we want from the database.
In this way, we display all the recommendations based on the city name we got from the user through our front end using Django.
2. What is NLP?
Natural Language Processing (NLP) is a concept where a machine will be able to understand human language and act accordingly. In our daily life, knowingly or unknowingly, we are making use of NLP. For example, speech-to-text detection software, etc.
3. What is a User Interface?
A User Interface is the framework that is visible to the user. It acts as a medium
between the user and the application. In order to design the front end, we usually make
use of HTML, CSS, BOOTSTRAP, JAVASCRIPT, etc.
4. Implementation of NLP
Let us have a look at the flow :
a) Execution of NLP in Django.
1. We will connect Snowflake with our Back end framework (Django in our case)
a) Installing Python Snowflake Connector.
//Install Python-Snowflake connector module for your python code using the below command in the terminal.
Pip install snowflake-connector-python
b) Connecting to Snowflake via Python Connector.
//connector code for snowflake to python
ctx = snowflake.connector.connect
(
user='ENTER USERNAME',
password='ENTER PASSWORD',
account='ENTER IDENTIFIER',
warehouse='YOUR WAREHOUSE',
database='DATABASE IN USE',
schema='SCHEMA IN USE',
role = 'ROLE IN USE'
)
Now that snowflake is connected with python code; we will be able to execute SQL queries.
2. Install LocationTagger Package for the python code using -:
- pip install location tagger
Why LocationTagger?
LocationTagger is used to identify the names of places in a given text.
After installing the module, import the module to the python code using:
- import locationtagger
3. Python code for displaying recommendations from snowflake.
cur = ctx.cursor()
sql = "select distinct(city) from PLACES"
cur.execute(sql)
records=list(cur.fetchall());
// In the above snippet, after connecting python with Snowflake, we were able to run SQL queries and also fetch the values in the form of a list of tuples( records). We fetched the city names from the ‘PLACES’ table defined in the connected Snowflake account.
sample_text = "I will visit Pune and Amritsar" // sample sentence
place_entity=[]
place_entity = locationtagger.find_locations(text = sample_text)
print("The cities in text : ")
print(place_entity.cities)
// In the above snippet we have considered a sample text containing Pune and Amrisar as places. There we used locationtagger module to find text using place_entity.cities.The output of this code will be [Pune, Amritsar]
def convertTuple(tup):
str = ''
for item in tup:
str = str + item
return str
// Above snippet is a function named convertTuple which converts tuple to string.
print("Database record include:")
dbrecord = [];
for row in records:
str = convertTuple(row)
db record.append(str)
// In the above snippet, we fetched the data from the database in Snowflake(all the cities fetched in dbrecord list).
Wildlife=[],Religious=[] ,Historical=[],Lakes=[] ,Missionaries=[] ,Artgallery=[] ,Stations=[]
Gardens=[],Malls=[],Touristattractions=[],AmusementParks=[],Churches=[],Theatres=[]
ScienceCentre=[],Galleries=[],Beaches=[],Waterfalls=[],MountainsHills=[],Schools=[]
Cemetery=[],Dams=[],Riverside=[],Academy=[]
// We have places to visit in different cities, and we also categorize the places. In order to display them separately, we form lists for different categories of places.
def my_fun(request):
for i in dbrecord:
if i in place_entity.cities:
print("Match Found",i)
cur.execute("SELECT NAME_OF_SPOT,TYPE_OF_LOCATION FROM places
WHERE city = %s",i);
records1=list(cur.fetchall());
print("\nPlaces to visit in " +i)
records1.sort(key=lambda x: x[1])
// In the above code, we have sorted places on the basis of location category.
for i in records1:
if i[1]=="Wildlife":
Wildlife.append(i[0])
if i[1]=="Religious places":
Religious.append(i[0])
if i[1]=="Historical places":
Historical.append(i[0])
if i[1]=="Lakes":
Lakes.append(i[0])
if i[1]=="Missionaries":
Missionaries.append(i[0]) if i[1]=="Artgallery": Artgallery.append(i[0]) if i[1]=="Stations":
Stations.append(i[0]) if i[1]=="Gardens":
Gardens.append(i[0]) if i[1]=="Malls":
Malls.append(i[0]) if i[1]=="Tourist attractions": Touristattractions.append(i[0]) if i[1]=="Amusement Parks": AmusementParks.append(i[0]) if i[1]=="Churches":
Churches.append(i[0]) if i[1]=="Theaters":
Theatres.append(i[0]) if i[1]=="Science Centre": ScienceCentre.append(i[0]) if i[1]=="Galleries":
Galleries.append(i[0]) if i[1]=="Beaches":
Beaches.append(i[0]) if i[1]=="Waterfalls": Waterfalls.append(i[0]) if i[1]=="Mountains/Hills": MountainsHills.append(i[0]) if i[1]=="Schools":
Schools.append(i[0]) if i[1]=="Cemetery":
Cemetery.append(i[0]) if i[1]=="Dams":
Dams.append(i[0])
if i[1]=="Riverside":
Riverside.append(i[0]) if i[1]=="Academy":
Academy.append(i[0])
// With the help of a loop we store values in the lists according to the categories of places.
context={}
context["key1"]=Wildlife context["key2"]=Religious context["key3"]=Historical context["key4"]=Lakes context["key5"]=Missionaries context["key6"]=Artgallery context["key7"]=Stations context["key8"]=Gardens context["key9"]=Malls
context["key10"]=Touristattractions context["key11"]=AmusementParks context["key12"]=Churches
context["key13"]=Theaters
context["key14"]=ScienceCentre context["key15"]=Galleries
context["key16"]=Beaches
context["key17"]=Waterfalls
context["key18"]=MountainsHills
context["key19"]=Schools
context["key20"]=Cemetery
context["key21"]=Dams
context["key22"]=Riverside
context["key23"]=Academy
return render(request, "nlp.html",context)
// In the above code snippet, we define a dictionary called context, and we generate a key for every category of place. We assign a value to each key, where the value is a list of places to visit.
Example: For category wildlife, we have assigned key1 and stored the listed wildlife( containing a list of wildlife-related places) as a value to key1.
// Then, we pass the context dictionary (which contains the results) as a parameter to the html page(Where we want to display results) using render.
b) NLP results at the frontend
<div class="container-fluid" style="margin-top: 50px;">
<div class="row">
{% for i in key1%}
<div class="col-2">
<div class="container-box" style="margin-bottom: 10px;">
<div class="content-box">
<h6>{{i}}</h6>
</div> </div></div>
{% endfor %}</div>
→ Given is a snippet to display a certain category of places (which were stored in the dictionary). We have used the jinja template to run a loop and obtain values in lists. In the same way, we can display all the categories using different keys created.
→ In this way, we can pass results to HTML pages from python code.
→ We can use CSS, Bootstrap and can display the output as per our choice.
5. Sample Results
We will have a look at how the implementation actually looks like:
Sample output of Textbox in UI where the user enters the text and the output after entering the text.
We have the output displaying category-wise places in a particular place.
By using different CSS classes, Bootstrap links, we can display the same recommendations in many ways as per our choice.
6. Conclusion
In this blog, we have learned how to connect snowflake with python, execute SQL queries in python which reflect the object of the snowflake database that we have connected, how execute NLP code in the backend, and finally, render the desired results of our NLP code in the frontend via Jinja templating.
7. References
1. For LocationTagger:
https://www.geeksforgeeks.org/extracting-locations-from-text-using-python/
2. Python code to HTML integration: