Archive for the ‘DBSprockets’ Category

DBSprockets is back, Baby!

Friday, December 12th, 2008

So I have had a pretty long hiatus from working on dbsprockets, but I’m back… with a vengeance.  So, I worked hard to get Rum working in TG2, but struggled my ass off and got nowhere.  Left tangled in a web of peak-rules that I did not want to decipher, I began to think about dbsprockets again.  I mean, were we really that far off from what RUM is offering?

The answer turned out to be no.  All I needed to do was to replace the dbsprockets primitives with a class structure.  This turned out to be about an 8 hour job.  Not bad for a day’s work.  And now you can do things like:

from dbsprockets.declaratives import FormBase
from myWebapp.model import User
...
class RegistrationForm(FormBase):
    __model__ = User
    __limit_fields__ = 'user_name', 'email_address', 'display_name', 'password', 'verify'
    __required_fields__ = 'user_name', 'email_address', 'display_name', 'password', 'verify'
    email_address = TextField
    verify = PasswordField('verify')
    __base_validator__ =  Schema(chained_validators=(FieldsMatch('password',
                                                        'verify',
                                                        messages={'invalidNoMatch':
                                                                  "Passwords do not match"}),))
registration_form = RegistrationForm()
class ATurboGearsController(BaseController):
    @validate(form=registration_form.__widget__)
    @expose('genshi:sproxtest.templates.register')
    def register(self, **kw):
        pylons.c.widget = registration_form
        return dict(value={})

This turns out to be a much simpler way of handling forms, because now you can subclass to your heart’s content (for instance, make one base user form which you subclass for admin, registration, profile and login), or even come up with your own wacky WidgetSelector that chooses widgets for you and subclass as you desire. Here is initial documentation, which I will express in more detail at a later date when I have more time to do it the right way.

The simple fact is that you can customize form widgets with ease, limit fields to a set of fields, drop a few fields, basically anything you can think of to change how the database schema actually displays on the page.  The same is true of field validators.  Simply define an attribute of the class that has a validator, widget instance, or widget type, and dbsprockets will do the right thing.  If you want to override both the field and the validator, all you must do is create a widget wit the validator attribute populated.  The greatest thing is that your forms (and tables!) will change with you as your database schema migrates.

Right now I am keeping the primitives way of getting the values from the provider to populate the tables/forms.   This is likely to change in the future.  The primitive way of doing forms is now deprecated.  I will be adding deprecation warnings to the code.  Hooray!

Look for ajax support in the near future on both the view and data side of things.  I have a clear understanding of ExtJS (2.0.1), JSON, and LGPL now and I am not affraid to use it.  In the mean time, a dev version of dbsprockets 0.5 is up at pypi.

Busy times with OSS

Friday, July 25th, 2008

I’ve not posted to this blog in over a month, mainly because I have been busy. Busy is a good thing I think. I have been helping clearwired get their newest web application off the ground, working on a number of screencasts, helping Mark with the latest batches of TG2 releases, learning about new and interesting OSS Python projects.

sphinx logo One of the more interesting things I have started to use with some frequency is Sphinx. Sphinx is a documentation system whichallows you to create webpages (among other things) with .rst files. It also grabs directly from your modules and inserts your docstrings (including your doctests.) It is great to have code which is not only documented, but also tested using nose. Look for a screencast from me in the near future on this one.

Two of the recent screencasts I have written show how to use virtualenv, PasteScript, and Nose. I go through a ground-up example showing how to create a virtualenv, a project package, and finally how to test that package and provide robustness with code coverage. The final screencast is about how to install TG2, which has become much easier in the advent of TG2’s first alpha release.

mountain gears TG2 is actually nearing it’s second alpha release. I think This is the best release yet. I spent some time moving DBSprockets to pep-8 compliance, and the two things TG2 depends on (tg.ext.silverplate and tgcrud) are now both using the newest release of DBSprockets. I also released basketweaver which allows you to create a simple local pypi made of static HTML files. Special thanks to Chris McDonough, who wrote the makeindex.py script. All I did was to fixed it up a little, and package using PasteScript, and provide a console_script for easy usage.

rum screenshot

Along with Sphinx, I have been trying to wrap my head around Rum, which is a new project to generate forms from database schema, much like DBSprockets. Although the internals of the system are somewhat mystifying, the API is squeaky clean, and in my mind what DBSprockets was eventually planned to be. The great news is that Alberto has done such a good job laying the groundwork for Rum I should be able to jump right in and apply lessons learned from DBSprockets. At this point it is safe to say that I will continue to maintain DBSprockets for bug fixes, but that the internals of it are going to be converted over to using Rum, before being deprecated altogether. DBSprockets had a great run, but it is time to move on and move forward with a superior design which promises to integrate so many WSGI technologies.

My Blog’s Wordle

Tuesday, June 17th, 2008

wordle

http://wordle.net/

Namespacing with Setuptools

Monday, June 16th, 2008

Lately I am finding that it is worthwhile to separate projects into their own packages, but since they are all in the same domain, I want to have them share elements for importing purposes. Enter namespacing, which I believe is a little-used feature of setuptools that people should take a serious look at.

What is namspacing? Well, if you are familiar with creating packages, you know that often times they share similar traits, which means it would be nice to have sort of a global package where they all reside, but then you would not be able to install the components of said package independently. Lets say we have a solar system package, and inside it are elements for each planet. You might import from them like this:

from solarsystem.earth import echosystem

from solarsystem.venus import atmosphere

But what if you didn’t want to package simulations of all the planets, because sometimes you just want to install one or two of them at a time. Namespacing gives you a way of creating packages which are related to each other, but do not clash.

First off, you are going to need setuptools if you don’t already have it. Download the script here and run it with python. This will install the ubiquitous setuptools package which contains easy_install.

The easiest way I have found to create a namespace package is to use zopeskel, which provides a basic_namespace template for your project. First, we easy_install zopeskel:

easy_install zopeskel

Now we can list the templates that paster provides. Paster is a tool which provides a developer with an easy way to create templates for packages, as well as other useful utilities for python file management. (Paster is automatically installed by zopeskel)

$ paster create –list-templates
Available templates:
archetype: A Plone project that uses Archetypes
basic_namespace: A project with a namespace package
basic_package: A basic setuptools-enabled package
basic_zope: A Zope project
nested_namespace: A project with two nested namespaces.
paste_deploy: A web application deployed through paste.deploy
plone: A Plone project
plone2.5_buildout: A buildout for Plone 2.5 projects
plone2.5_theme: A Theme for Plone 2.5
plone2_theme: A Theme Product for Plone 2.1 & Plone 2.5
plone3_buildout: A buildout for Plone 3 projects
plone3_portlet: A Plone 3 portlet
plone3_theme: A Theme for Plone 3.0
plone_app: A Plone App project
plone_hosting: Plone hosting: buildout with ZEO and any Plone version
plone_pas: A Plone PAS project
recipe: A recipe project for zc.buildout
silva_buildout: A buildout for Silva projects
At the top of the list is basic_namespace, which we will be using. For example, lets make a solarsystem.mars package:

paster create -t basic_namespace solarsystem.mars

Paster will then prompt you with a series of questions, of which the first two are the most important:

Selected and implied templates:
ZopeSkel#basic_namespace A project with a namespace package

Variables:
egg: solarsystem.mars
package: solarsystemmars
project: solarsystem.mars
Enter namespace_package (Namespace package (like plone)) ['plone']: solarsystem
Enter package (The package contained namespace package (like example)) ['example']: mars

After you answer the rest of the questions, it will create a directory structure that looks something like:

solarsystem.mars/
|– README.txt
|– docs
| `– HISTORY.txt
|– setup.cfg
|– setup.py
|– solarsystem
| |– __init__.py
| `– mars
| `– __init__.py
|– solarsystem.mars.egg-info
| |– PKG-INFO
| |– SOURCES.txt
| |– dependency_links.txt
| |– entry_points.txt
| |– namespace_packages.txt
| |– not-zip-safe
| |– paster_plugins.txt
| |– requires.txt
| `– top_level.txt
`– zopeskel.txt

At which point, you are ready to create a development install of your project. Simply change to the directory:

cd solarsystem.mars

and:

python setup.py develop

Code for your new module would go in the solarsystem/mars/ folder, inside the new package. Any python you have installed this new package will
now be able to:

import solarsystem.mars

Namespacing is a great way to organize your work into digestible morsels, which can be installed one by one as needed by your perspective users. It is a great way to take advantage of all of the tools which have been created to make distribution and versioning easier.

Sprints, Vacation, Refreshment

Thursday, June 5th, 2008

It’s been a while since I have blogged for a few reasons. One, I took a vacation. No work, open source or otherwise for 11 days. I barely even checked my email. One thing that is great about living in Colorado: I don’t have to go anywhere on vacation, and my family will come to me! I took my sister up the First Flatiron, my dad fishing in Cheeseman Canyon, and all of us took a nice stroll through Chautauqua Park. We ate out almost every night and enjoyed catching up since I have moved 2000 miles away.

Previous to that, I managed to win a skirmish in the framework war at work, and spent an obscene amount of time (on and off the clock) bringing my first production-level TG2 site. This was a really fun project because my coworker and I were given the reigns and let loose on implementation. In 3 hours we had a working TG2 site with data in an EXT grid, with database schema expressed as a tree. In 3 weeks we had a complete reporting system (using EXT) with object-based security. We were also able to implement a file upload manager, and a select shuttle (for assigning groups to secure objects) thanks to the help of Sanjiv Singh, my GSoC protege. Thanks to all of you who have helped myself, and TG2 get off the ground.

One of the reasons I was so absent from the OSS community was that I was trying to wrap my head around what was going on with the EXT licensing, which changed under my feet as I was developing our in house app. The move from LGPL to GPL/Commercial was a shock to the community, and in my opinion based primarily on one person’s greed. My personal choice is to finish up whatever EXT support I have to do for our in-house application, and never look back at it again. I will be moving on to Dojo, and have advised my GSoC students and other people involved with TurboGears to do the same.

SilverPlate Demo site Now that I am back from vacation/new project hell, I have been able to release a new module for TurboGears, tg.ext.silverplate, which is a plugin for TG2 providing User Management and Profile pages which are customizable. This all fits under the TGTools domain, which has become a home for tg.ext.repoze.who (Authentication for TG2) and tg.ext.geo (An upcoming library for Geographical support in TG2). If you get a chance, you might want to check out the new TGTools googlecode site.

I also did a new release of tg.ext.repoze.who, which provides authentication/authorization to TG2. The only functionality I as able to contribute was a change to the name-spacing, as well as a full test of it’s functionality. Right now I am working on an LDAP bridge which will allow your TG2 site to authenticate against LDAP, and then dump you into the TurboGears Domain for all of the authorization stuff (groups, permissions, etc.) I should have this completed in a few days. Most of the work for tg.ext.repoze.who was done at the last world-wide WSGI-Turbogears sprint. It looks like TG is going to do a release in a few days.

This weekend I am headed to ABQ for some sprinting with clearwired on their next-generation web application. I am hoping this time around TG2 is mature enough in their eyes to take advantage of everything it has to offer, since they have currently chosen to work simply with Pylons as a framework. Either way, I will be working with some of the best people in the business, and we should be able to push TG foreward one way or another.

TurboGears and Google Summer of Code

Wednesday, March 12th, 2008

TurboGears is undergoing a monumental effort to participate in GSoC.  Ok, maybe not monumental, but at least 5 of the developers have been hard at working putting together an application even Google wouldn’t turn down. Even if we are not accepted, we are planning on participating by way of PSF, as we have done in previous Google Highly Open Participation contests.  We have developed a number of ideas which students can choose from, or students are welcome to come up with their own TurboGears ideas, and I am sure that one of our mentors will be able to match up with you.      

My own ideas revolve around DBSprockets and TwTools, which is not surprising since I am the owner of said projects.  The largest project and the one which I have the most desire to see put into action is that of a TurboGears CMS.  There had been some work done last year by the guys at Pagoda which produced a brilliant screencast, but the project seems stalled out and it would be nice to see it revived.  Furthermore, the solution I proposed is intended to be much more modular, so you could pick apart portions of the CMS and put them in your existing applications.  I think this would be the most flexible solution, and also one which would employ much that DBSprockets, TwTools, and Toscawidgets have to offer.  If you are a student who is interested in working on this project, please don’t hesitate to drop me a line.  You can also track me down at Pycon for the remainder of the week, and at the TurboGears sprint next week.  If you are a student who is eager to get started feel free to participate in our sprint, remotely or in person.   

One of the great things I see coming from this mini-project is that we now have a very nice set of concrete ideas about how to make TurboGears better.  Whether or not students participate in the development we still get a huge benefit from the creation of ideas, and it gives the development team and possibly new developers a target to make TurboGears the best it can be.  My hope is that these ideas will not only bring a students to our project, but also bring some developers out of the woodwork who may have started similar projects and would like to contribute.  The bottom line is, if you aren’t a contributor, and want to be, here is a great place to start.

Announcing DBSprokets 0.2 Release Candidate

Sunday, February 17th, 2008

I am pleased to announce the first DBSprockets 0.2 Release Candidate. This release comes shipped as 100% code-coverage tested for all API modules (non-dbmechanic).

Many thanks to my beta-testers and developers who have submitted patches, especially Nathan McBride, Michael Brickenstein, and Jason Kirtland. Thanks to Alberto Valverde, who’s work with Toscawidgets is invaluable to this project.

The biggest part of v0.2 is probably Primitives, which give developers an easy way to generate web content from database definitions. If you have a sql database, and want fast web content, this might be the
ticket. A simple few simple calls to table reflection with SQLAlchemy, and you can use makeForm, makeTable, and getTableData to generate web content.

The second notable module in this release is the DBMechanic, which allows you to do all of your database crud with about 3 lines of code added to your Controller.

Supported on this release are TG1.0 and TG2.0-preview with both Primitives and DBMechanic. Primitives have also been verified in Grok, which took a certain amount of work with respect to wsgi and Toscawidgets. Mysql and sqlite are supported with this release Posgres has been tested and is shown to work with dbsprockets.

There are no known Issues to date.

Everything you need to know to get up and running with DBSprockets is here:
http://code.google.com/p/dbsprockets/
Thanks,
chris

Python Frameworks [in] compatibility.

Wednesday, February 13th, 2008

This is a response to Mark Ramm’s post entitled: “Site Components” in Django and TG2 .

First off I wanted to commend Mark for his insightful post. Mark certainly has considerable perspective on both frameworks, and has a great ability to divulge the best of both. He is often pointing me in directions so that I might make DBSprockets better. One day I even received a link to a Ruby on Rails application, ActiveRecord. He certainly can think outside of the box when it comes to solving the world’s Python Framework dilemmas, and is not afraid to express himself openly about his opinions.

“TG2, like Django will define a set of tools that can be used in building re-usable web site components. TG2 users should be able to powerful, reusable components, with SQLAlchemy, Genshi, ToscaWidgets, and the whole TG2 toolchain. ”

–Mark Ramm

I am so glad that Mark pushed TG2 in this direction. I am glad that he pushed DBSprockets in this direction. This week I
worked on getting DBSprockets to work within a Grok application. With not too much effort I was able to get Genshi, SQLAlchemy, and Toscawidgets working within a Grok environment. DBSprockets followed suit. I was amazed at how little work it was to get Toscawidgets working in Grok despite the complexity with which it interfaces the web framework. Granted, I did have to get Grok working through WSGI, and I have Repoze to thank for that. But in the end, all I had to do was easy_install the correct packages, and modify 4 lines in the .ini file, and poof, Toscawidgets in Zope. Who’d a thought a year ago (before the Pylons/TG “merger”) something like this was possible?

So what is my point? When I started out, I asked Mark for commit writes to TG so that I could build DBSprockets into it. The response was, “Well, no go off and do the google code thing and get back to us.” What a great move that was, because now I am able to support many more frameworks and have a much broader user base. And Django… you’re next.

Turbogears Worldwide Sprint

Monday, February 4th, 2008

The last TG2 sprint was seen as a resounding success, despite the fact that coordinating such an event is quite a challenge.

On February 23rd Mark Ramm will be visiting Colorado and plans on coordinating yet another sprint. I will probably still be working on dbsprockets which is likely to be a very important component of the new framework. If you are a python developer who is interested in Turbogears and would like to contribute, don’t hesitate to contact me.

Pycon08 Registration is open!

Monday, January 21st, 2008

I’m very excited to attend this year’s Pycon . In past years, pycon has been a place for me to learn about the multitude of open-source products that are available, and see how to use them. Turbogears, Twisted, even Iterators were topics that I tackled during previous visits to Pycon. I am certain that this year will be no different.

This year, my third time attending the conference I was unable to find a tutorial that spiked my interest, so I decided to help Mark Ramm and Ben Bangert on their tutorial on both TG2 and Pylons, which I have been participating in the development of. Mark told me that DBSprockets is going to be showcased as a competitor to Django’s newforms. I definitely don’t want to miss that. Mark and Ben are both very knowledgeable about Pylons and TG, and their tutorial should be an excellent way to learn how to use these cutting edge technologies.

I have found that one of the best ways to learn new things is to sprint with other developers. I will be sprinting on TG for this year’s Pycon, and I encourage anyone who has not sprinted before to join a sprint for the topic of their choice. You will learn more than any tutorial or talk can teach you, and best of all you get to contribute.

So, hopefully I’ll see you there. Drop me a line if you are looking for a climbing partner at the conference, I usually try to start up a birds of a feather for those who are vertically addicted.