XHTMLl 1.0 Strict and CSS Validated Drop-Down Menu

I needed to create a few navigation systems for a new web site skinning application I was working on. After testing a few I found googling I ended up writing my own that is Simple; Passes CSS and XHTML Validation; Gracefully Works Without CSS; and isCross Browser Compatible. I had tried a few of the other one's I'd found using Google and ran into major incompatibility issues in the different browser I had installed: Chrome, FireFox, IE7 and Safari for Windows. There are two display issues when using IE. If you are using IE7 or greater you will have a little extra padding below the navigation, this doesn't effect functionality but slightly changes the look of the overall page when displayed in any of the other tested browsers. If you are using IE6 or less the drop down doesn't work so your menu can be a link to menu area navigation page or a little JavaScript should be added to support IE6 or older. If you would like me to accommodate either of these IE situations please let me know and I'll look into it. You can view my initial development version of the first navigation system at: http://threshold-creations.com/navigation.html
You can use view source to see everything it's all in the single document.

Django Templating System and Skinning

There are several reason why implementing a skin system for your web site is advantages. If you're user centric you can you can do it because some users enjoy being able to customize the web sites they use frequently. I use skin for development purposes so I can turn on/off features such as style sheets and JavaScript or so I can develop a template and test it on a site before making changes to existing templates. Additionally every site I've been at for any length of time want a new look and feel. With a skin system you can implement all of these without having any down time for your average user and also maintain your earlier look because some users will always prefer it at least for a while, it's a comfort thing. I have documented and implemented such a system at Project.Django-Development.com.

Django Full Text Search with Whoosh

Public subversion or browse dd_search source

svn co http://projects.django-development.com/svn_public/dd_devel/trunk/dd_search

This is a full text search engine based on Whoosh.


Overview

Using Whoosh the dd_search application will add full text searching to any project. It does require a minimal amount of effort when defining model classes so that the developer can limit what is indexed and what is not in the event you have some secure information you don’t wish to be queried by an anonymous user. Additionally it allows for dynamically created indexes so you can break your indexes into security zones or simply by department etc. It supports searching multiple indexes and indexing records to multiple indexes. The rest of this section goes into more detail on the subject.

Initially I was interested in making this implicite instead of explicit but realized quickly that was to risky and would potentially open to many security concerns. So following along the Pythonic mindset I chose to make indexing of models and columns explicit while minimizing the amount of work required to accomplish the feat. The configuration consists of two entries per indexed model and a single declaration in your setting.py. At least my initial design is satisfied with this level of configuration.

The easiest way to handle secure searching is to define different indexes for different types of data. Your public data can be searchable by the world so I suggest making a public index. If you additionally have internal or other higher security groups I suggest that you create additional indexes for each security area. I would never index critical columns that have credit card numbers etc. because that's one more vulnerable location to secure and not what this tool is intended for. Additionally you may want to index by other factors like data type, i.e. accounting, human resources etc. This is all accomplished with a single entry when you declare your model class by specifying dd_search_index = "index,index,index".

Additionally not all fields should be indexed because some will have very sensitive information while other may contain data that is so common as to dilute the effectiveness of a search. Whatever your reasoning you must choose the text based columns to index. This is the second and only declaration that is required to start building indexes for any model. This again is accomplished with a single entry when you declare your model class by specifying dd_search_fields = "field,field,field"

The other configuration requirement is to define a directory where you indexes will live. This needs to be writable by the account your web server is running as which is another reason to not index really sensitive information. In you settings.py file define DD_SEARCH_INDEX_DIR="/full/path/to/directory"

You should also declare the get_absolute_url and unicode methods for each model as well. Although not a requirement it make things work better since this is what is used to generate the search results.

Methods

update_index

You will not call this method it's called by the post_save signal. So every time a record is saved it will see if it needs to be indexed. It returns as quick as possible to minimize overhead if no index is defined for the model. This method lives in the models.py file so that it is loaded when django initializes before any records are save. It does the following:

1. Return right away if an index and fields are not defined for the instance.
2. Create the main index directory defined in setting if it's missing.
3. Concatenate the indexed fields into a long string for indexing.
4. Loop through indexes.
1. If index doesn't exist create it.
2. Update/Add index data


dd_search

The dd_search method takes two parameters, a csv list of indexes and the query. It will return the combined search results. This is intended to be called by your secured search routines that manage what indexes can be searched by whom.

search_internal

I also provide a secured search_internal method as an example to get you started. This simply has the login_required decorator on it to verify that a validated user is searching.

Whoosh Schema

The schema stores the relevant information to identify the specific django record it is associated with. The content is a concatenated string of the indexed columns since which column contained the information is not relevant.

WHOSH_SCHEMA = Schema(
app=STORED(),
model=STORED(),
pk=STORED(),
title=TEXT(stored=True),
url=ID(stored=True, unique=True),
content=TEXT(analyzer=StemmingAnalyzer()),
tags=KEYWORD)

I have additionally added tags to the schema in the event I add tagging to the indexing in the future which seems like a logical step.

Recursive Relationship

This information is also available here.

I frequently run across a situation where I need to group objects. Often these objects will belong in more than one group and the groups will consist of smaller groups. I use two Many to Many fields to represent this situation. In this example the employees field is related to my Employee model while the relationships field is related to itself creating our recursive relationship. Neither field is required and for the recursive relationships field I turn symmetry off. So here are the models.

from django.db import models

class Employee(models.Model):
    name = models.CharField(
            max_length=32)

class Department(models.Model):
    name = models.CharField(
            max_length=64)
    employees = models.ManyToManyField(
            Employee,
            null=True)
    departments = models.ManyToManyField(
            'self',
            null=True,
            symmetrical=False)

Now that we've defined our models we can create a function to return a query set of the employees for a given department. This is a simple function that creates a query set containing all of the employees for a given department and then calls itself doing the same for each of the sub departments and so on. This allows for unlimited depth and nesting of groups/departments.

def employees(department):
    query_set = department.employees.all()
    for department in department.departments.all():
         query_set = query_set | employees(department)
    return query_set

That's all there is to it. I typically create a utils.py file in my applications to store such functions. I added a few more utility functions to help demonstrate functionality as well. If you are a visual person you can use the dump function to return the organizational chart in no specific order.

def dump():
    from recursive.models import Department
    results = ''
    for department in Department.objects.all():
        results += "%s\n" % department.name
        for employee in employees(department):
            results += "\t%s\n" % employee.name
    return results

It's often nice to know if a object is part of a group so I also have an has_employee function for testing this.

def has_employee(employee, department):
    if employee in employees(department):
        return True
    else:
        return False

And here are some tests to make sure it's doing what I think it's doing.

from django.test import TestCase

class SimpleTest(TestCase):
    def test_recursive(self):
        """
        """
        from recursive.models import Department
        from recursive.models import Employee
        Employee(name='Jay').save()
        Employee(name='Brad').save()
        Employee(name='Aaron').save()
        Employee(name='Mark').save()
        Employee(name='John').save()
        self.assertEqual(Employee.objects.count(),5)

        emp = Employee.objects.all()

        d1 = Department(name='President')
        d1.save()
        d1.employees.add(emp[0])

        d2 = Department(name='VP Finance and Administration')
        d2.save()
        d2.employees.add(emp[1])

        d3 = Department(name='Computer Services')
        d3.save()
        d3.employees.add(emp[2])
        d3.employees.add(emp[3])

        d4 = Department(name='Finance')
        d4.save()
        d4.employees.add(emp[4])

        d5 = Department(name='Finance and Administration')
        d5.save()
        d5.departments.add(d2)
        d5.departments.add(d3)
        d5.departments.add(d4)

        from recursive.utils import employees

        self.assertEqual(len(employees(d1)),1)
        self.assertEqual(len(employees(d5)),4)

        from recursive.utils import has_employee
        self.assertEqual(has_employee(emp[0], d1), True)
        self.assertEqual(has_employee(emp[4], d1), False)
        self.assertEqual(has_employee(emp[4], d5), True)

        from recursive.utils import dump
        self.assertEqual(dump(),"President\n\tJay\nVP Finance and " \
                "Administration\n\tBrad\nComputer Services\n\tAaron\n\t" \
                "Mark\nFinance\n\tJohn\nFinance and Administration\n\t" \
                "Brad\n\tAaron\n\tMark\n\tJohn\n")

        print "\n%s" % dump()