I just read Mark Cuban's eBook 'How to win at the sport of business', which is worth reading btw, and stumbled upon a real gem:

I recently read and article by Paul Kagan referring to George Gilder's "vision" that in the future TV will die, regardless of delivery medium, simply because people will watch only what they want to watch.

How wrong he is. Why he is wrong is a lesson in basic business.

It war Aaron Spelling, I believe, who said, "TV is the path of least resistance from complete boredom." Which is another way of saying that it's easier to watch TV than to sit there and do nothing.

Thinking about that, I quickly thought of a lot of businesses/business-models/... which are based  on the assumption that people tend to choose the path of least resistance. While TV is a prime example, there seem to be a lot more ways to make good money from it.

This list is not meant to be exhaustive, so if you find more examples, feel free to add a comment or two (some of these might be specific to Germany)! Also, please beware that I do not judge or condemn any one of the examples. I just try to point out phenomenons which are built on man's tendence to laziness.

1. TV

As mentioned in Cuban's book, people, in need of something to do, choose watching TV over idling because it is easier to achieve. While a bit shocking, this explains a lot about the success of many very questionable TV formats. No choice to make, just turn it on and be numbed out.

2. Free trial subscriptions (which become real subscriptions if not cancelled)

Most likely everybody at one point or another encountered this: promotion people try to talk you into a free 2-week subscription of your local paper. After the trial runs out, you get a regular subscription if you do not cancel it in time. Apparently, for some people, it is easier to just pay the ongoing subscription instead of cancelling the trial.

3. Fastfood

No, it is not healthy. And no, it is not even especially cheap. But fastfood offers a TV-like advantage in the nutrition department: it is easier to eat fastfood than to prepare yourself something real to eat. After filling up on burgers, you just don't have the energy for anything more than watching TV. 

4. Facebook

Of course you could just use *diaspora. After long hours of persuasion, you could probably talk a lot of your friends into switching as well. Why don't you? Because Facebook is just so darn easy to use, and it already has all your friends. No what did that crazy guy just say about privacy?

5. Privacy

If you really wanted to, you could use the internet in a way that most people or enterprises would have a hard time creating a profile of you. You would use your own mailserver, you would disable cookies, javascript, flash and set your browser to maximum security settings. Why don't you? Because it is too damn hard. Some services do not work, you miss out on a lot of stuff. It is just easier to give up your private data.

6. Phone-contracts/Banks/Insurances

A lot of those companies make you jump through endless hoops if you want to discontinue their products/services or switch to a competitor. Form after form has to be filled out and sent to different destinations. Most likely even in the shape of actual paper. If played out correctly, the companies succeed in making switching more difficult than staying with the worse product or service.

7. Groupon

Not only do they offer amazing prices, they offer amazing opportunities to kill time. If you are searching for stuff to do, Groupon delivers with interesting enough opportunities at very little cost. Would you really want to go play paintball for 30 Euros per hour? Probably not, but since you save 50% (regularly it's 60! Oh my god, I just saved 30 bucks!) and got your afternoon all planned our for you, why the hell not?

8. Amazon

This example only works on a couple of levels, since Amazon is really consumers' best option for a lot of stuff. If they offer the best price, where else would you buy? But still, they are not the cheapest for everything. Do you buy there nonetheless? Of course you do! All your data is already there, you profit from awesome shipping and you do not even have to input your payment data again. So you overpay a couple of times, who wouldn't when it is this convenient?

Recently I discovered deck.js, a very cool framework for creating HTML presentations. I really liked how slick it looks, and after further inspection, the HTML code turned out to be quite simple. Just put something into a <div class="slide"></div> and you got yourself a slide.

That got me thinking: What about a tool, that converts some simple text format into a nice presentation? I looked into a couple of things, and it turned out markdown was up for the task. It basically is a text format that allows for some text formatting. Further research unveiled the Python markdown module, which not only converts markdown into HTML, but also allows the user to write custom extensions for document processing.

Long story short, after a little trying I wrote an extension, that takes a markdown document, and converts it into a nice set of slides. Every time the parser encounters a H1 or H2 element in the markdown, a new slide begins.

Example:

# Titlescreen

## First content slide
Some content

## Second slide

* a list
* with
* three items 

creates a 3-slide presentation. Check out the project at Github.

Let's consider a fairly standard website built in Django/Rails/Drupal/FancyNewFrameworkXY. You have a navigation which allows quick access to the most important features of your application. Each click on a link in your menu reloads the page and renders the appropriate location for the user. Nothing spectacular here. If you want to increase the performance of your app, you have a couple of possibilities. Some are easy to do, like enabling caching of static resources. Some are harder, like optimizing database queries. And some require extensive restructuring of your app to cope with the increasing scale that a lot of happy customers bring to your site. The last one is really a luxury-problem, but nevermind.

Today I have a small suggestion to increase the responsiveness of your application by using a nice little trick dealing with how your site is displayed. Instead of reloading the entire site with every click, we will instead use Ajax to partially reload only the needed parts. And to make it even cooler, we will make that completely transparent, so you do not have to mind which way is used in your application. I will use Django to demonstrate how this works in a very smooth way. Check out the (very basic for clarity purposes) code at Github.

To complete this objective, we need to pass through a series of 3 steps:

1. Organize your view-layer by creating a template hierarchy

If you embrace DRY, chances are you already are doing this. If not, please embrace DRY. In this instance it means that you should separate the visual structure of your site from the content of the actual views that do something. In Django this is easily done using template inheritance. Instead of duplicating a lot of HTML code in the template for each view, we just say

{% extends "index.html" %}

in each.

2. Differentiate between standard vs. Ajax requests and serve the appropriate response

The next step is to actually detect if someone requests the page traditionally through a link or if it should be served as Ajax. Luckily, most browsers send a specific header to signal the server that the request is some kind of Ajax magic. So we should be able to determine what's what. But wouldn't it be cool to do so in a special place in our application? Enter Django template context processors. As suggested by their name, they process the context supplied to templates for rendering. You activate one or more processors in the settings of your project. While Django supplies a number of built-in processors, it is super-easy to write your own: It just is a python function that takes a HttpRequest object as parameter, and returns a dictionary of new template context variables.

We can leverage that into a simple logic: Whenever the request is done via Ajax, we try to render our views in the exact same way, only we select an empty parent-template. The processor looks like this:

Now we only have to use that knowledge in our templates, which is even easier: We just edit our templates and change

{% extends "index.html" %}

into

{% extends root_template %}

Notice the missing hyphens? This means we use the template variable instead of a hardcoded value. For normal requests, it still uses index.html as template, so nothing changes inside our app.

We need to add the ajax.html template so that we can render the Ajax requests as well. It only contains a content-template block, since the rest of the HTML will already be loaded through prior requests.

3. When loading the site, change the behaviour of the menu

Now lets wire everything up, so that actual Ajax requests are made. For this, I recommend using a Javascript framework, I will do it with jQuery, but of course others work as well. We only need to edit our original root-template for that. Once the document is fully loaded, we select every link in the menu-div and change its href-attribute and onClick behaviour. Reload and try to click a menu link. Voila.

Conclusion

I would like to emphasize a couple of points:

1. We did not make any internal changes to the app itself to make this work
2. The old URLs still work, so your SEO efforts are not voided
3. Pretty awesome, right?

A couple of days ago I read an article by Harry Brundage: Today, Web Development Sucks. In his posts he talks about the state of a lot of web applications, and some of the problems. One of which, the one that resonted with me most, was code duplication. I suspect every developer has come across one or another form of it in his or her career. The duplication most prominent in the post concerns itself with form validation on the web.

See, user input is a tricky business. You cannot assume anything about what your users will enter in the forms you give them, do not assume understanding or common sense. Somebody always finds a way to work around all your assumptions. So the only way to ensure correct form data is to use proper validation of input. You let the user enter whatever the hell they want, but before you commence whatever you do, you validate the input. If all lights are green, go ahead. If not, show the user the form again, and prefarably tell them what they did wrong (I know, just the last part done correctly would be amazing).

But what is the big deal? In today's era of ever more complicated (web) applications some people dare to expect a minimum standard of usability. And in case of forms that means that everytime the user clicks submit and receives an error message, the less likely he or she is to actually complete the process. Which brings us back to form validation: Since you cannot trust anything the users enter, you definitely have to check the data at the server. There is NO way around that. To guarantee a decent usability you also should validate while the user enters data, or at least prior to submitting. Harry's concern now was that this leads to a lot of unnecessary code duplication, since you perform identical checks at the server- and the client-side (probably using different programming languages, but nonetheless).

This got me thinking: Why not specify the validation once at the server, and use that specification to generate javascript code which performs the client-side validation automatically!

I sat down a couple hours and put together a really barebone solution for django and jQuery (find code at Github).

What I did was the following:

1. Subclassed the django.forms.Form class to add the client-side validation
2. Wrote some own validator classes which specify the server- and client-side code to validate
3. When rendering the forms, the validation code is automatically generated from the validators of the formfields. The corresponding javascript is appended to the form the client sees.
4. Currently the validation happens at the javascript keyup event, which means after every keystroke the user sees if the field is correctly filled or not (invalid fields are marked by a special CSS class)

Let's quickly look at how to make it work:

1. Define a form

# Import the classes
from validforms.models import ValidForm
import validforms.validators

# Subclass ValidForm instead of django.forms.Form directly
class TestForm(ValidForm):

    # Define the fields, using the validators from this app, supplying the regex as
    # a parameter
    # The max_length validator is supported implicitly, since it is very common
    field1 = forms.fields.CharField(validators=[validators.RegExValidator('[0-9]+')],
                                    max_length = 100)

    field2 = forms.fields.CharField(validators=[validators.RegExValidator('[a-z]+')],
                                    max_length = 10)

2. Display the form. Assuming you have prepared a view, your templates should look somehow like this:

<html>
  <head>
    <style>
      .invalid { /* Use this for invalid form fields. Customize at will */
        background-color: #FF8888;
      }
    </style>
    <script type="text/javascript" src= "https://ajax.googleapis.com/ajax/libs/jquery/1.5.0/jquery.min.js"></script>
  </head>
  <body>
    <form>
      {{ f.as_p }}
    </form>
  </body>
</html>

As you see, you almost don't need to do anything. Just define the style for your invalid fields, import jQuery and display the form using the regular django methods.

3. Write own validator

# This is one of the existing validators
class RegExValidator(Validator):
    """ Subclass to validate arbitrary regular expressions
    """
    def __init__(self, re):
        self.re = re

    def _validate(self, text):
        """ Method used for form validation by django """
        return re.match(self.re, text) != None

    def client_side(self):
        """ Returns javascript validator for client """
        return """function (x){ return (""+x).match(/^%s$/) != null; }""" % (self.re)


class DateValidator(RegExValidator):
    # We just use the existing regex class
    def __init__(self):
        super(DateValidator, self).__init__('[0-9]{4}-[0-9]{2}-[0-9]{2}')

Granted, this does not really validate a correctly formed date, but for arguments sake I won't dive deeper. I am pretty happy about this little proof of concept and will probably continue to build this. If anyone has suggestions I would be happy to hear them. For now a roadmap might be:

1. Somehow make validation more "semantic" (dependent on server-side data)
2. Abstract from jQuery to allow integration with other JS frameworks
3. Port to Rails

When you use Dropbox to share files among several users, let's say org-mode files, you can run into the problem that two people simultaneously save different changes, especially when those files are generally open in your editor. Emacs supplies auto-revert-mode, which realizes when file changes occur on disk, and reverts the file automatically. Problem solved. The following snippet activates the auto-revert-mode for all files in your dropbox folder (path may have to be modified.)

To execute the Freemium model successfully is a science of its own. Even more can one appreciate the rare occurrences of good implementation of the paradigm. Vimeo represents such a case. Their product, video hosting, is essentially free. Everyone can use the service, you have some quotas which you cannot exceed as a free user. Premium accounts do not impose these strict quotas on you. This in itself is not something worth mentioning.

What makes it more interesting is their implementation of the sharing process: After uploading the video, the user will see something like the following image:

But still, there has to be a way to make that a little less ugly...

Being an avid Django user for quite some time now, I cannot help but develop ever growing and complex applications. Since Django makes it really easy to distribute whole systems or even smaller parts, I try to keep that advantage in mind. The latest project consists of several apps, which are able to run on different systems and are interconnected through some API calls. While this minimizes coupling and allows for maximum flexibility, I ran into an annoying thing:

Django's Testing Framework

Don't get me wrong, I really like they way they handle testing, mostly it suits my purposes just fine. But as my project got more and more components, I began to wonder how to test the whole mess efficiently... Since the standard unit-testing is somehow magic it works well for one project. But if you have to connected projects which depend on each other, things get messy. Let's say that Project 1 needs Project 2 set up an running to function properly. So every time we run 1's tests while 2 is not running, we encounter errors.

The quick fix is to let an instance of 2 run before we start testing 1, but this is really not scalable. Just imagine having 12 tightly connected projects, with multilevel dependencies, setting everything up manually.

I began thinking about possibilities to circumvent all of the troubles. Then it dawned on me that the situation had an easy way out: a testing wrapper, that allows you to specify dependencies for your Django project, and sets up these dependencies prior to the tests so that everything works well, and tears them down afterwards. No more manually running utilities projects, no more forgetting that the data of the dependent application was stale, no more worries.

After two hours of learning some things about Pythons subprocesses, I finally got a draft working. The usage is quite simple:

• Create a YAML file specifying your projects and their dependencies
• Call the testwrapper instead of ./manage test
• Everything works fine

The solution

projects:
  login:
    dir: /home/ulf/django/login
    port: 8080
    reset: [loginapp]
    fixtures: [base]

  payment:
    dir: /home/ulf/django/payment
    port: 8081
    fixtures: [base]

  awesomeapp:
    dir: /home/ulf/django/awesome
    deps: [login, payment]
    tests: [app1, app2]

Looks simple enough, doesn't it? In this example I have one app, awesomeapp, which depends on two other applications, login and payment. Since I want to test awesomeapp automatically, I have to specify its dependencies with the information which is needed to run them properly. We need to specify the port on which the application shall run (awesomeapp expects the login app to be at port 8080), the Django apps which shall be reset before starting and the fixtures that are to be loaded. In the future we might get some extensions on that.

So now what happens?

I call "deptest.py config.yaml awesomeapp", the script starts by reading the config and determining that awesomeapp has two dependencies. Now both dependencies are started using the specified variables. The fixtures are loaded and apps are reset. Now the tests for awesomeapp are started, you can specify which tests to run with the "tests" variable. The tests are run using Django's standard command so you see everything on the screen that happens. After the tests are done, the dependencies are torn down.

Gimmicks

You can supply self- or even circular dependencies, no problem.

Get it here

Github, PyPi package is on todo-list

Alex Holt's nice article showed how to get Nginx, Memcached and Django to work together. Out of curiosity I wanted to try that setup for myself, with one modification: instead of using fastcgi to serve Django I wanted to employ uWsgi. In case you did not read the original article, he describes how to setup an environment in which Nginx directly queries Memcached for responses, which were generated and cached by Django. Only in case of cache misses is Django used to generate the result again. While surely not suitable for every usecase, it has a certain appeal to deliver computational heavy resources from a cache and directly through the webserver, minimizing overhead. I will not go into details about that here, since Alex did that very well.

Let's look at the main location handler which resides in the server-section of the Nginx config file:

location / {
            if ($request_method = POST) {
                    proxy_pass http://localhost:9004;
                    break;
            }
            default_type  "text/html; charset=utf-8";
            set $memcached_key "/your-site-name-$uri";
            memcached_pass localhost:11211;
            error_page 404 502 = /django;
    }

    location = /django  {
            proxy_pass http://localhost:9004;
            break;
    }

He uses the Memcached integration to query the key for the requested resource, which directly derives from the project and the URL which shall be retrieved. After construction of the key, he passes the request to Memached, which then returns the contents, if found. If not, we encouter an error, in which case he alters the location.

This part became tricky to replicate, with Fastcgi and the accompanying use of

proxy_pass

, we can just go ahead and pass everything on to the proxy instance. The Fastcgi config is a lot more extensive, see the article for details. So I tried to use uWsgi instead, first we have to define the django source in the server config, I use socket configuration in this instance:

upstream django {
    ip_hash;
    server unix:sock/uwsgi.sock;
  }

(Look here and here for details on how to setup uWsgi with Nginx and Django.) After setting it up I ended up experimenting a lot with different uwsgi_params settings, none of which produced the result I wanted. Everything messed up the process, mostly through redirect that got not handled correctly by either Nginx or Django. After some more research I finally found something interesting: named locations, which allow you to make a "symbolic" rewrite in Nginx:

error_page 404 502 = @fallback;

Using this kind of redirect, I could suddenly just specify the fallback handler to integrate the uWsgi part:

location @fallback {
      include     uwsgi_params;
      uwsgi_pass  django;
    }

The end result of the config file looked like this: