Wednesday, September 25, 2013

Crash Course in Django - Part 2 : Step by Step Guide to Make Django App Real Quick


Introduction :  In  first part we covered the basics of creating objects , saving and retrieving them from database. In this series we will try and extend it further with a user interface and setting it all up -Lets try and create one employee listing ( Refer part 1 for the Employee Problem ) interface where all the companies employee are listed and can be searched by name. This will a step by step tutorial to make a django application rapidly and will also try and integrate various other technologies like Jquery , Twitter bootstrap etc with Django Templates. Lets start again with the bullet points:

Tip : 
  • There are two ways to handle this ride
    • You can get the code from here and start walking through each point  to make it real quick. With all files in place you can just cruise through 
    • Or you can start step by step with thought process and start creating your files and verifying the changes made with mock ups and gist file changes posted after every essential step
  • Now which step to choose depends - again on how quick is quick. 
  • Last , please don't forget this is not a series on Jquery , User Interface or html layouts etc. they are just there to fit the purpose 
Get Geared: 
  • Lets create a mock up first  - how our interface will look like 

  • For creating this form we would be using twitter bootstrap  and the corresponding html will look as shown below

  • In real world application I would remove the side pane of employees listing with link button for detail as we can show all the employees present with click of search button with no parameters. The side pane is included in this article to show some of the Django template language features. 
  • Lets quickly start creating the project structure 
    • Create Project : $ django-admin.py startproject demoapp
    • Add Postgres option in settings.py as database
    • Add demo app as in INSTALLED_APPS section of setting.py
    • Add models.py file to demo app sub directory and add the Employee class we discussed in part one. The screen shot below shows the setting.py changes we made

    • From the command prompt in the Project directory where manage.py file is present call
      $ python manage.py syncdb and we will see the message like  which will also create a table in the database. When we call syncdb for the first time django asks us to create one admin login - create one. How to use admin login we will cover later in the series for  now lets focus on the task at hand



  • Now if we want to check if  things are OK , lets learn a new utility from the command prompt type the following: $ python manage.py shell . This will open a command prompt in context of the current project. So if want to check the database connection type 
    • $ from django.db import connection
    • $ from django.db import connection & cursor = connection.cursor(). If no error then we are able to connect to the database
  • We can similarly use shell to import Employee object and ORM functions to check the status of our code. For example
    • $ from demoapp.models import Employee will import Employee
    • Employee.objects.all() will return blank as we don't have and employee yet
  • Lets add some Employees from the console itself
    • emp = Employee(employeeid=124,name='Mathew',dob='1980-1-11',salary='5423.4') & emp.save()
    • emp = Employee(employeeid=1004,name='James Holmes',dob='1970-10-21',salary='7000') & emp.save()
    • emp = Employee(employeeid=14,name='Monica Hilton',dob='1986-07-10',salary='3000') & emp.save()
    • Employee.objects.all() will now return a set of 3 employees
  • We can do a batch save but lets leave it for advance use cases
Lets try and get our html page up & running now with our back end almost ready lets hit on layout part
  • What kind of URL we want to expose to our Employees - something like below will do for now
    • url we need : http://127.0.0.1:8000/employees
    • Lets add  url(r'^employees/',get_all_employees) the URLS.py file  and corresponding get_all_employees view to views.py file. The changes will look as shown below for both the files 
                    
  • But before we load our html some changes we need to make to settings.py file and we need to have some understanding of how Django Template Engine work
  • As we know Django view returns a HttpResponse object and we also tried passing string and JSON to it.  How about giving one well constructed html ? We can do that using Templates . So create a html file and replace the dynamic content - like user name , in our case employee name with variables with double braces {{variable_name}}
  • For example my Html says <p>Welcome James !<p>  where James is the name of logged in user and will keep on changing for each logged in user we assign a variable to this and change it like  <p>Welcome {{USER_NAME}} !<p>  and we will pass o a USER_NAME variable to the Template from our view using render_to_response function. render_to response is standard django function that takes the name of the template - html file as first argument and dictionary as second argument. In the above example the render_to_response will be like render_to_response('User.html',{'USER_NAME':Computed User Value})  and the  Django Engine when renders it will replace USER_NAME in html to the value that we have passed. If you want to read more on this please refer here .
  •  For our rapid learning purpose whatever  values we want to pass or show in html pass them via render_to_response django function as KEY-VALUE pairs . Use the KEYS on html file which will be replaced by values while rendering. This html file is our Template. 
  • Two more things that can come handy while creating Templates - if / else statements on the passed variable can be used within the html file like
    • {% if VARIABLE_1 == TEST_VALUE  %} <p> Good Results </p> {% else %} <p>  Bad Result </p> {% endif %}. Django Will now selectively render html based on the TEST Value and VARIABLE_1 comparison which we pass from backend
    • {% for item in item_list} <p> item.attribute_name </p> {%endfor%}. This will be used to parse list of object and accessing the value inside that object by dot operator. As previously said we will pass this item list also from backend
  • So in our case we will pass employee list from backend like render_to_response('Employee.html',{'employees_list':employees_list})  and parse it in the html file for creating the employee list. The side pane of employees we see in the html gets generated like: 
       
  • Please note the way we are passing Employee Id to Javascript function - This will be used to fetch employee detail once employee is clicked
  • Last setup change - how will Django know where to pick the template from - there is a TEMPLATE_DIRS  tag in settings.py and there we will give the path of the folder where we will keep our Employee.html  , which is  TEMPLATE_DIRS "demoapp/templates" . And  for loading the static file one needs to provide STATIC_URL = '/static/'  and append the <script> </script> , link style sheet  load  directives in html as below. Don't forget to create one static folder in demoapp folder just at the level we have template folder and put the bootstrap folder ( js, css, img folders inside it). So django engine will look for static folder as defined in STATIC_DIR and append the paths to load the javascript , css and other resources 
    • Also add load static files directive to tell Django Engine to do so {%load staticfiles%}
    • <script src="{% static "bootstrap/js/bootstrap.js"%}"></script> 
    • <script src="{% static "bootstrap/js/jquery-1.7.2.js"%}"></script>
    •  <link rel="stylesheet" type="text/css" href="{% static "bootstrap/css/bootstrap.css"%}"/>
  • So create on templates folder inside the demo app folder and place the Employee.html file there
  • Try running the server with python manage.py runserver command ( runs the application in default development server )and type http://127.0.0.1:8000/employees/  on the browser . The html we made will show up. Along with the list of Employees we have in the system
         

  • We now have two problems to solve - 
    • On click of employee Employee detail should render on the right pane
    • Implementing employee search
  • For first we will implement get_employee_by_id view on  the server. Map this view with corresponding URL and call this URL via jquery GET function and then parsing the returned data to be placed in the form. ( As this is a Django Series -Not talking much about the JQuery bit here , please Google about JQuery Ajax calls. Please note how we mapped the URL in Jquery Ajax call to the urls.py file. See onEmployeeClick function below for javascript code 
  • For second - Searching employee by id is same as getting name of employee and other details we discussed above via employeeid. I will talk more about Search by name because searching by Id will always return one employee but seaching by name might return more so search functionality will try and place result in row order using twitter bootstrap. Please refer the code below for the same.See onEmployeeSearch function below for javascript code
  • And now the mapped url and views changes :
  • Now we can download the whole code and compare the changes  from the repository here
  • Some files  come by default when django creates an app like init.py and wsgi.py. The later one is used in deployment scenarios that we will cover later ..
This brings us to end of this part and I guess we are good to go ....




Monday, September 23, 2013

Crash Course in Django - Part 1 : Django Basics

Introduction :  This is first  lecture for people who want to learn Django rapidly. This assumes some familiarity with Programming of Web based application but not mandatory. Here I will try to be as generic as possible and will try and redirect the reader to specific as and when needed.  And as true for all the matters of interest - there is Google. So lets start bullet point approach to quickly get us going for Django based web development.

Gear Up : 
  • Install Django :  Install Django as mentioned in the Documentation here. The installation is fairly straight forward both on Windows and Linux based OS.
  • You need a Data Base - Django comes with SQLLite Database for practice purpose but I suggest you either install MySql or Postgres. If you are not using the inbuilt SQLLite database you also need to install connectors for same. Find more about connecting to various databases here. I am not repeating it here because all this is fairly well documented
  • When you are up and running with basic setup of Django start by creating a new project by typing the following command on the command line:  
django-admin.py startproject demoapp
  • Demo App is a name of the directory and will contain manage.py file. This is the file that is used by Django to deliver its magic. ( More on this later). The command will also create a directory structure as 
demoapp/
    manage.py
    demoapp/
        __init__.py
        settings.py
        urls.py
        wsgi.py
        models.py ( This is file we will add later. Not by default)
        views.py ( This is file we will add later. Not by default)   
  • Django Follows MVC pattern to achieve separation of concerns. Like the way we have Controllers in the MVC pattern to take care of logic, Django call the controllers as views. These views are collated in one file named as views.py. So any function you can think of like addition of two integer or a function to sort and return certain data set, will be written in views. View is nothing but a function that takes certain argument -request and givens certain output - HttpResponse
  • The constrain with view is that it should always return HttpResponse type of object. So even if we want to return a string or JSON, put it inside a HttpResponse object like 
  • As shown above view has a default argument as request - this is nothing but a HTTP request object which can be used to access variables from URL or from the html form being submitted. Also it returns HTTP Response. In a nutshell use the request object to get the data out of html form. Use the HttpResponse to render data on the form. Django comes with a template language which can be used to render calculated variables on the form. Will be covering Django Templates in later part of this series
  • Next file of interest is urls.py , this file is nothing but a collection or a map of url and function ( Django view ). Its a way to tell the Django Engine - if a URL of specified type ( this also includes regular expressions) comes as a request on the server , redirect it to the view specified in the map. From there on the view takes care of processing the HttpRequest object and rendering a HttpResponse
  • Essentially URL.py is a file where pattern in URL is matched and appropriate function is called from the views.py file. So if the incoming request is like http://127.0.0.1:8000/get_units , Django Engine will match it to  first URL entry and will call get_units function from the view which will in-turn return the JSON object we talked about earlier
  • So till now - views.py contains  logical bundling of function that operate on Incoming Request  and urls.py  helps that request to reach the suitable function. Question is where and how we get and manipulate data from ? After all its not only about rendering the incoming request to a Django Template
  • Comes in the frame models.py file - This is the file that contains all the data holders we need to get and manage data from database system. A model is in-program representation of Data Tables in the Database. Lets take an example on this - consider the application for creating a Employee List
  • Employee List application will  store employees detail in the database - lets assume our employee has four attributes and attribute type
    Attribute NameAttribute Type
    Employee IdInteger
    Namestring
    Date of Birth Date Time
    SalaryFloat
  • In the Database this will be employee table & in django this will map to employee model as 
  • For legacy databases - use db_column attribute to map each and every table column with Django Model and use Meta tag to map the table as shown above. For fresh implementations call python manage.py syncdb and Django will create all the tables for us
  • Once the model is created we can use Django ORM to directly work with Employee Objects , which in-turn will interact with database. For Example if we would like to do following operations on our Employee from our python view ( remember all the logic and action is in views.py) import the employee model in the view and call the CURD functionality directly using Django ORM. 
    Creating Employeeemp = Employee(employeeid=124,name='Mathew',dob='1980-1-11',salary='5423.4')
    Saving emp.save()
    Get All employeesEmployee.objects.all()
    Get Employee By idEmployee.objects.get(employeeid=124)
  • Once we have reached so far we can easily manipulate on data sets in data base using Django ORM to put it all together lets try and see how one will implement a URL -  http://127.0.0.1:8000/get_employee?EMPID=124  API using whatever we have discussed where EMPID is passed a Query string parameter to the view
  • Last - settings.py is the file where one keeps all the configuration info regarding the web application - like data base settings . logging , application to include , admin module etc. 
  • This brings us to the end of this part. Hope we are good to go ! 

Wednesday, September 18, 2013

Business Rule Engine in Python


Problem:  This a typical business case – we have a lot of business rules to define a bill for certain Business Entity. These business rules were changing every now and then. Just to give a brief example, what are the kind of business rules we are talking about:
Sample rule set:
Charge $ X if quantity P is between A & B
Charge $ Z if time of day is between D & E
Charge $ Y if quantity Q is between B & C
Give DISCOUNT if R is less than F

These Rules will repeat in sets for various other business scenarios and Categories for example
Category 1
Rule 1

Rule 2
Category 2
Rule 1

Rule 2

Rule 3

Rule 4

One way to solve this issue is a lengthy business analysis cycle, where a Business Analyst sits for 2-3 month and write a complete set of business rules for each and every case. The problem with this approach was the Category and Rule keeps on changing, so by the time the Analysis is documented and given to development team the rule itself might have changed. Still let’s assume this will not happen so the code that will come out will be something like this:


Now its clear that the values a_multiplier_1 ,v_1, v_2 will be changed  for the sake of business rules frequently and code will have lot of if and else - Such code is error prone and hard to maintain , patchy stuff and don't forget the business problem of collecting these requirement

Solution:  

Well one way to solve this as above -With python we can solve this a bit differently by using eval(expression) built in function.  But first as defined in Python Docs

eval : The expression argument is parsed and evaluated as a Python expression (technically speaking, a condition list) using the globals and locals dictionaries as global and local namespace


>>> x = 1
>>> print eval('x+1')
2

Steps below uses the logic above to accomplish the solution for the problem

  • Lets hand over the power of writing & changing the business rules to Analysts team, let them update the values using forms - using Django its a trivial job.
  • The form will allow the Business Analysts to create variables and assign them values 
  • These variables will be of two type - Constants & Expression as shown below
      
  • Then allow user to write expression like  -
Variable_1  
100
Variable_2
200
Variable_3
Variable_1 + Variable_2
Variable_4
IF Variable_3 > 1000 RETURN VARIABLE_3* VARIABLE_1 END
IF Variable_3 < 1000 RETURN VARIABLE_3* VARIABLE_2 END


  • Based on the order of the calculations  write the code that stores these variables and passes the variables of type expression ( Variable_3 & Variable_4) to the eval function to be processed as Python expression. 
  • Lets think of Logic to Calculate Variable_4 ,assuming the function parsing the variables has already stored Variable_1   , Variable_2 and Variable_3 , say in a dictionary with Key as variable name and value as calculated values                                        


  • Return all the values calculated as JSON , which will get consumed by any other function as Key Value pair.  
And with this we are good to go - writing code to write no code ! 

General: Many people will like to call it as Domain Specific Language where in Domain Related Parser can  be written in English Language as shown above and the code can parse it for domain specific purpose. It makes one think how will languages like Java and C# handle these dynamic expressions. That might be a  topic for some other day I guess.