Fernando Correia’s Weblog

October 16, 2008

Editing Python on Ubuntu Linux

Filed under: Software Development — Fernando Correia @ 9:45 pm
Tags: , , ,

Still using Eclipse with PyDev for Python… Let’s face it, Visual Studio with ReSharper (for C#) is so much better…

Anyway, I tried GVim, but it is too old-fashioned. I’d rather use Notepad++ if it were available on Linux. I even tried JEdit again, after so many years…

But Eclipse is doing the trick for now. I expect that NetBeans with Python support will be my platform of choice for Python when it is released.

September 3, 2008

Django 1.0 is Out

Filed under: Software Development — Fernando Correia @ 10:11 pm
Tags: ,

Jacob Kaplan-Moss announced the release of Django 1.0. That’s a great milestone for this Python Web framework.

Jacob recalls how the project grew to a proportion that they couldn’t imagine in the first days, and concludes that

“Django 1.0 represents a the largest milestone in Django’s development to date: a web framework that a group of perfectionists can truly be proud of.”

Among the major features in the release notes are:

  • Stability and forwards-compatibility
  • A new documentation site
  • Re-factored admin application
  • Improved Unicode handling
  • An improved ORM
  • Automatic escaping of template variables
  • GeoDjango
  • Pluggable file storage
  • Jython compatibility
  • Generic relations in forms and admin
  • INSERT/UPDATE distinction
  • Split CacheMiddleware
  • Refactored django.contrib.comments

August 31, 2008

Plone

Filed under: Software Development — Fernando Correia @ 2:23 pm
Tags: , ,

I have another website to build. Last time I used Drupal, and it is certainly a good tool. Since I am learning Python I went out to find a good CMS in that language and I found out that Plone is a major one.

It is really an advanced tool. It’s easy to install, have a powerful user interface and can be extended in Python in an object-oriented way.

There is a page with some great screencasts that show what Plone can do. I recommend the second one, Editing with Plone. It is 12 minutes long.

After seeing it, I decided to spend some time to learn more about Plone.

August 29, 2008

Utility Mill

Filed under: Software Development — Fernando Correia @ 7:19 pm
Tags:

Almost every day I’m amazed to see how Python inspires people to build great stuff.

One example is Utility Mill. It is an application that lets you easily build Web utilities. These utilities have Web pages so you can interact with them. But they also have a RESTful API so they are actually Web services.

The utility can accept input in the form of parameters, process this input with arbitrarily complex logic written in Python, and output the results in several formats like text, HTML, XML, JSON or files.

I recommend the brief video tour and the Maze Generator as an example.

Now, that’s a quick way to publish a Web service if I’ve ever seen one!

August 27, 2008

A PHP developer looks into Django

Filed under: Software Development — Fernando Correia @ 9:15 pm
Tags: , ,

Thierry Schellenbach has published a very interesting article about his study for selecting a Web framework. A PHP developer that values the Symfony framework, after weighing the issue along with a colleague he decided that for many projects Django and Python are a better fit than Symfony and PHP.

This is not a typical holy war rant. He points where Symfony has advantages over Django. But concludes:

Both are great, but Django more so.

August 25, 2008

Get out of that sandbox

Filed under: Software Development — Fernando Correia @ 2:15 pm
Tags: , ,

Brian Clapper just published an excellent article about Making XML-RPC calls from a Google App Engine application.

He demonstrates how we can live with and work around limitations of a sandboxed environment like Google App Engine.

I also liked the way he shows how the good design of xmlrpclib makes it easier to adapt it to other needs.

August 23, 2008

Example of RIA in the cloud

Filed under: Software Development — Fernando Correia @ 12:34 pm
Tags: , , , , ,

When I decided to learn and get some first-hand experience about Rich Internet Applications (RIA) and distributed application hosting (cloud computing), I chose to do it using Flex and the Google App Engine. Flex is a mature tool that has a plethora of rich interface capabilities and is very good to connect to services on the Web. And the App Engine is a very intriguing proposition where you use Google’s infrastructure to run your application.

To share my experience and get some feedback I started a project called “Flex and Python Test”. Now this project is finished and released. The application is a very simple project portfolio that can keep a list of projects and participants. The data is kept on the client for fast response, and the modifications are sent to the server for persistence. This is what the application looks like:

The source code for this application is available on github. If you want to understand it, download it and browse the code. To help, I will explain a bit about the design.

This is the general architecture:

The client is a Flex application running in a browser. Flex allows us to have a rich interface without dealing with incompatibilities between browser implementations. The server is a Python application running in Google App Engine. The client send requests through HTTP using the AMF encoding format. This is a binary format that is very efficient for Flex applications. The server processes the requests and sends responses also encoded in AMF. The server exposes its interface in the Remote Procedure Call (RPC) pattern.

If you download the source, take a look at README.txt. It explains how to run and deploy the application. There are three directories: flex-client contains the Flex application that will run in the browser. python-client contains an alternate client in Python that you can run in your computer to initialize data, test the service and learn the API. server contains the Python service that is deployed to Google App Engine.

To build the Flex client I decided to use a Model-View-Controller (MVC) framework. A sound structure helps when you want to build larger applications. After experimenting with Cairngorm, I went for PureMVC. If I would make another iteration I might now try Mate.

With PureMVC, this is the overall architecture of my client application:

The Model represents the data. It uses the Proxy pattern to control access to the remote service. The View contains the user interface. It uses the Mediator pattern to implement Supervising Presenters that manage the interface components. The Controller contains business logic. It uses the Command pattern to execute actions that interact with several other components. The model, view and controller objects communicate using a Publish/subscribe-style Observer notification scheme.

For instance, this is how the Participants proxy asks the server to insert a new participant:


public function addParticipant(participant:Participant):void
{
    ServiceGateway.GetConnection().call("ProjectParticipantsService.save",
        new Responder(onAddParticipantResult, onFault), participant.toDto());
}

On the server side, Google App Engine takes care of executing your application when it receives requests. The Python application uses PyAMF to decode the client requests and execute the service operation that is being called. The data that is transferred between the client and the server is stored in Data Transfer Objects (DTOs). Since both Python and Actionscript are dynamic languages, I didn’t declare formal type structures for these DTOs. They are just dynamic objects. The service operations use App Engine’s Datastore API to store and retrieve objects.

This is how the ProjectParticipants service executes the operation to insert a new participant:

def save(self, participant_dto):
    if hasattr(participant_dto, '_key') and participant_dto._key != None:
        return self.update(participant_dto)
    else:
        return self.insert(participant_dto)

def insert(self, participant_dto):
    participant = ProjectParticipant()
    participant.project = Project.get(participant_dto.project_key)
    participant.name = participant_dto.name
    participant.put()
    return self.to_dto(participant)

Some of my previous posts have more details about how I built this application:

I’ve deployed this sample application to Google App Engine. You can interact with it using this URL:

http://fernando-correia.appspot.com/

Click on Edit projects to open the RIA client. If you refresh the homepage you should see the updates you made.

I hope this can help other people who want to learn some of these techniques and tools. Your feedback is most welcome.

August 13, 2008

Flex with GAE: adding participants to projects

Filed under: Software Development — Fernando Correia @ 9:45 pm
Tags: , ,

I’m making progress on my endeavor to build a Flex client to a Python service hosted on Google App Engine.

On the last iteration I had the whole architecture set up, using PureMVC on the client side and PyAMF on the service side. The application was able to show, create, edit and delete projects. It also handled project participants, but only in memory, without persisting them.

Now, the project participants are being persisted as Google App Engine model objects, referencing the project model objects.

This is the model definition on the server side:


class ProjectParticipant(db.Model):
    project = db.ReferenceProperty(Project, collection_name='participants')
    name = db.StringProperty()
    created_at = db.DateTimeProperty(auto_now_add=True)
    modified_at = db.DateTimeProperty(auto_now=True)

The service operations are implemented using a standard PyAMF RPC pattern (not REST yet):


class ProjectParticipantsService:
    def get(self, key):
        logging.debug('get %s' % (key))
        return self.to_dto(ProjectParticipant.get(key))

    def insert(self, participant_dto):
        logging.debug('insert %s' % (participant_dto))
        participant = ProjectParticipant()
        participant.project = Project.get(participant_dto.project_key)
        participant.name = participant_dto.name
        participant.put()
        return self.to_dto(participant)

I am using the anonymous DTO approach that I described in a previous article:


def to_dto(self, participant):
    if participant is None: return None
    dto = DTO()
    dto._key = str(participant.key())
    dto._id = participant.key().id()
    dto.project_key = str(participant.project.key())
    dto.name = participant.name
    dto.created_at = participant.created_at
    dto.modified_at = participant.modified_at
    return dto

On the client side I have a domain object representing a project participant:


[Bindable]
public class Participant
{
    public var _key:String;
    public var projectKey:String;
    public var name:String;
    public var created_at:Date;
    public var modified_at:Date;

    public static function fromDto(dto:Object = null):Participant
    {
        var participant:Participant = new Participant();
        participant._key = dto._key;
        participant.projectKey = dto.project_key;
        participant.name = dto.name;
        participant.created_at = dto.created_at;
        participant.modified_at = dto.modified_at;
        return participant;
    }

    public function toDto():Object
    {
        var dto:Object = new Object();
        dto._key = _key;
        dto.project_key = projectKey;
        dto.name = name;
        dto.modified_at = modified_at;
        return dto;
    }

    public function toString():String
    {
        return name;
    }
}

I also follow PureMVC’s approach of using proxy objects to manipulate the model objects, like this:


public class ParticipantsProxy extends Proxy
    {
        public function getProjectParticipants(projectKey:String):ArrayCollection
        {
            if (participants[projectKey] == undefined)
               participants[projectKey] = new ArrayCollection;
            return participants[projectKey] as ArrayCollection;
        }

        public function addParticipant(participant:Participant):void
        {
            ServiceGateway.GetConnection().call("ProjectParticipantsService.save",
                new Responder(onAddParticipantResult, onFault), participant.toDto());
        }

        private function onAddParticipantResult(result:Object):void
        {
            var participant:Participant = Participant.fromDto(result);
            var projectParticipants:ArrayCollection = getProjectParticipants(participant.projectKey);
            projectParticipants.addItem(participant);
            sendNotification(ApplicationFacade.PARTICIPANT_ADDED, participant);
        }

        // (etc.)
    }

The rest is done on the View layer, like this:


private function onAddParticipant(event:Event):void
{
    var newParticipant:Participant = new Participant();
    newParticipant.projectKey = participantsPanel.project._key;
    newParticipant.name = StringUtil.trim(participantsPanel.newParticipant.text);
    participantsProxy.addParticipant(newParticipant);
    participantsPanel.newParticipant.text = "";
}

You can download and browse the source code through github. Feedback is welcome.

August 2, 2008

Returning DTOs from PyAMF

Filed under: Software Development — Fernando Correia @ 12:14 pm
Tags: , ,

Serializing Google App Engine model objects using the AMF protocol is very efficient. AMF even supports references to objects, so repeated serialization of references to the same object won’t waste bandwidth.

For instance, if we have a model like this:

class Project(db.Model):
    code = db.IntegerProperty()
    name = db.StringProperty()
    department = db.IntegerProperty(default=0)
    created_at = db.DateTimeProperty(auto_now_add=True)
    modified_at = db.DateTimeProperty(auto_now=True)

class ProjectParticipant(db.Model):
    project = db.ReferenceProperty(Project, collection_name='participants')
    name = db.StringProperty()
    created_at = db.DateTimeProperty(auto_now_add=True)
    modified_at = db.DateTimeProperty(auto_now=True)

And we return a list of 5 participants of 2 projects, each project’s data could be serialized only once, although on the client side we will have the project data in each participant instance. This is good and PyAMF supports this feature. [Edited after some clarification from Nick Joyce.]

But sometimes we may want more control over what is serialized. For instance, we can have an object that has references to 5 other objects. But our client may not need to use their data, so it would be a waste to serialize it.

We can have fine control over what is serialized, with more work on the server side, by creating a data transfer object (DTO) that will be returned. That way we can decide whether we want deep or shallow serialization. We can also restrict the data that will be serialized and return keys or references as we think will be better for use on client side.

This is an example of the DTO approach:

class DTO(object):
    pass

class ProjectParticipantsService:
    def get(self, key):
        return self.to_dto(ProjectParticipant.get(key))

    def to_dto(self, participant):
        dto = DTO()
        dto._key = str(participant.key())
        dto._id = participant.key().id()
        dto.project_key = str(participant.project.key())
        dto.name = participant.name
        dto.created_at = participant.created_at
        dto.modified_at = participant.modified_at
        return dto

A generic DTO class is used. I don’t see much point on creating a specific class, like you would do on Java. This way, you only have one place where you control your DTO’s structure. DRY.

Some caveats that may bite you when you are coding late at night:

  • Your DTO cannot be a direct instance of object, like dto = object(). Instances of object cannot be extended with dynamic attributes.
  • Be sure to convert the key to a string. If you try to serialize the key itself, like dto.project_key = participant.project.key(), there will be an error like “AttributeError: ‘str’ object has no attribute ‘iteritems'”.

I published an example at github.

“At last! It works.” (Cosmix)

July 25, 2008

A few explanations about my test project

Filed under: Software Development — Fernando Correia @ 8:15 am
Tags: , ,

I thank Rien for giving valuable feedback on my test project. He had some questions that show I am not explaining my approach well enough. So I will try to clarify. Let see:

Love your idea of starting a good commented pytthon-flex-cairngorm project! Very much needed indeed!

Glad to hear you think that too. I think that as we share what we learn, all of us will benefit. One thing, though: this project is about a Flex client to Google App Engine. Not necessarily about Cairngorm. I will probably go the way of PureMVC. No problem for Cairngorm users, anyway, because the project is about the client-server integration, not about the framework for the Flex client.

I got your project running, there’s only one thing I do not understand just yet.

You have 2 folders “server” and “python-client”. They contain each pyamf in its whole? Is that necsessary?
It would help me and others much if you could explain why they are splitup in 2 seperate folders.

I agree that is a waste. Space is cheap, though. My reasoning is that I want my “server” directory to contain only what must be running at Google App Engine. No client tools at all. That is, I want a self-contained web service on GAE, and nothing else.

Since I also wanted a Python client to be able to test PyAMF in a way that is independent of Flex, that led me to create a “python-client” directory. I put a separate copy of PyAMF there so both subprojects (Python server and client) would be independent and self-contained, with no external dependencies.

And how the client.py knows of ProjectService.py and EchoService.py I don’t seem to get how they are linked up?

Well, first about the server. It does not know about any client. It only knows about requests that come through PyAMF.

About the clients, both (Python and Flex) are bound to the server in a naïve way that is suitable only for a test project. They use a fixed URL to the server (http://localhost:8080/) and they use the name of two services (EchoService and ProjectService) that are routed inside main.py in the server to the actual implementation classes.

Because I only start up “dev_appserver.py server”? And then openup the flex app.

You can start only one instance of the server on port 8080 and have both the Python and the Flex client talking to it.

I hope that clarifies a bit. Feel free to ask if you have more questions. Just keep in mind that I am still learning all this.

Good luck!

Next Page »

Create a free website or blog at WordPress.com.