New Instructor Dashboard

I’ve been working on a new instructor dashboard the last week using Plotly + Dash. Its taken a while to wrap my head around the declarative style, and to learn the ins and out of plotly. But I’m getting there. Let me know if you want a closer look and would like to give me feedback! In the meantime here is a preview.

Screenshot 2022 12 30 at 1 40 59 PM

adventures with flask-cors

  • I want people to be able to write and host the static parts of any book on any server. You can think of each page in a book as its own single page application.

  • I want to provide back-end services so that students using any book hosted anywhere can save their programs and answers to quizzes etc.

  • I want to continue to gather research data on how students learn computer science.

  • I want to make the registration and login process as easy as possible.

Since the static parts can be hosted anywhere (including a site like interactivepython.org) The interactive parts are going to involve making cross-domain XMLHttpRequests (xhr). Of course the first thing that happens when you have a page hosted on static-site that makes an xhr request to ajax-server is that you get an error. Browsers and sites work together to disallow cross-domain requests to prevent a variety of nasty behaviors. But, there are many times (wlike now) when you have a legitimate reason for doing this. So, the w3c created the Cross Origin Resource Sharing (CORS) standard to help developers get around this. Cory Dolphin has created an excellent plugin for Flask developers called Flask-CORS. The plugin is a great example of the brilliant design behind Flask and in fact the entire WSGI stack.

The Really Simple Approach

The first thing you find when you start googling about this problem is that there is a seemingly simple solution. If you have control over your AJAX response you simply need to add an HTTP header Access-Control-Allow-Origin: * problem solved. Now everyone in the world can make xhr requests to your server and use the results in their page.

Adding a header is pretty simple in Flask. All you need to do is use response.headers.add("Access-Control-Allow-Origin", "*" Problem solved, moving right along to the next programming challenge.

Or maybe not. Minutes later you realize that this is not all that great because you have decorated some of your requests to require a login. That wont be a problem if the static page is served from the same domain because you will automatically get the session cookie, and the Flask-Security extension will eat that cooking and validate things for you. BUT if your static page is not served from the same domain you will not even get the session cookie. Oh Bother. But you also have a second problem. You have probably violated the CORS specification without even meaning to. Really, if I had to read the spec for every web standard I wanted to use I would seriously think about changing careers. But, here is the important part you may not return a CORS header unless the request contains an origin header! Chances are you tested you change with a quick curl call to your endpoint, saw the Access-Control header and were happy. But you sure didn’t give it an origin header on the request when you did that. So to summarize, we have two problems we need to solve:

  1. We want to incorporate authentication into our cross origin strategy.

  2. We want to be good citizens and follow the spec.

The Smart Approach

The smart approach is to use a nice extension where other people have figured this out, and presumably followed the specification. Enter Flask-CORS. You can enable CORS support with a simple decorator @cross_origin This will automatically add the Access-Control-Allow-Origin: * to responses. As long as your test request includes an Origin. If you are like me you will forget that part, and then wonder why the extension must not be working. So this solves problem 2.

To solve problem 1 here is a snippet of code that works just fine.

@ajax.route('/ajax/page')

@login_required

@cross_origin(supports_credentials=True)

def test():

    return jsonify({'foo':'bar'})

The above responds to the url /ajax/page I have all of my API calls in an ajax blueprint with ajax as part of the url. I’m requiring that the user is logged in before I allow them to access this endpoint. I also want it to be allowed cross origin. This is where the parameter to the @cross_origin comes into play. Supports credentials sets up the CORS response to return an additional CORS header: Access-Control-Allow-Credentials: "true". For one final twist, you need to know that when you have supports_credentials=True you may NOT set Access-Control-Allow-Origin: * You need to be specific and set the origin to the origin that comes in the request headers. To Make this work and try it out from the client side, here is a bit of HTML/Javascript.

<button onclick="corsTest();">TestCORS</button>



<script>

    function corsTest() {

        var xhr = new XMLHttpRequest();

        xhr.withCredentials = true;

        xhr.onload = function () {

            alert(xhr.responseText);

        }

        xhr.onerror = function () {

            alert("error");

        }



        xhr.open("GET", "http://example.com/ajax/page", true)

        xhr.send()

    }

</script>

Note that you need to set xhr.withCredentials in order for your session cookie to be sent along. By default cookies are NOT sent with cross origin requests.

Now, I may end up adding more to this as I discover the intricacies of so called “Non-Simple” requests. That is requests beyond simple GET and POST, as I work on moving my API toward a RESTful API which uses PUT and others. This will nodoubt enlighten me about preflighted requests. Which I can only assume means something different than sitting around in an airport bar waiting for your flight to be called.

There is a lot more detail and background on using CORS at the following two sites:

writing a runestone lab the easy way

As part of the grand reorganization of the various tools and software associated with the Runestone Interactive project I am planning to also write a series of tutorials to help people get started. The major aspects of this reorganization are discussed in detail in the Project Roadmap, but for the sake of some context I can summarize the major efforts as follows:

  1. Separate the distribution and development of writing tools from the server.

    1. Make a pip installable runestone package.

    2. Remove the interconnectedness between the components and Sphinx. In other words support the user of runestone tools in environments like Markdown, and even wysiwyg html editors.

  2. Re-architect the server side focusing on services for the writing tools

    1. Create an authentication service that supports CORS for cross domain AJAX use.

    2. Create a standard REST API for logging and storing student data

  3. Create a web application (or integrate with another) for grading

Since I just completed Part 1.1 I thought it was a good time to talk about how easy it is for you to now use the runestone tools for creating a lab for your students, or lecture notes and presentation materials for your class.

Getting Started

The major steps in getting started are

  • Installing Python

  • Installing the runestone tools

  • Building your first lab

Install Python

  1. If you are on a Mac you are already done with this step.

  2. If you are on Windows you will need to go to Python.org and download Python3.x. The windows installer is a typical installer and you can just click your way through it.

If you are an advanced Python user you may want to may want to create a virtualenvironment for this project but it is not a requirement.

If you are on Windows you may want to edit your PATH environment variable following the instructions here. Again, mac users can ignore this.

Installing the Runestone Components

You are going to need to use the command line for the rest of this tutorial, so start up a Terminal (/Applications/Utilities on a Mac or run PowerShell or cmd.exe on Windows) I will repeat myself here. These commands need to be run from the command line, not from the Python shell.

Run the pip command

$ pip install runestone

Or on Windows if you have not modified your PATH try:

C:\\Python34\Scripts\pip.exe install runestone

From now on I’m only going to give the Mac way of running the commands. If you are on windows you will need to add C:\\Python34\Scripts to the beginning of the command and add .exe to the end.

You can watch as a lot of text goes scrolling by. But as long as you don’t get any errors you should be good to go. You only need to do these first two steps once. Once you have installed Python and runestone you will not have to do it again.

Starting your first Runestone Project

Here is a session of me on my computer creating a simple project.

$ mkdir mynewproject

$ cd mynewproject

$ runestone init



This will create a new Runestone project in your current directory.

Do you want to proceed?  [Y/n]: y

Next we need to gather a few pieces of information to create your configuration files

Project name: (one word, no spaces): myhello

path to build dir  [./build]:

require login   [false]:

URL for ajax server  [http://127.0.0.1:8000]:

Your Name  [bmiller]: Brad Miller

Title for this project  [Runestone Default]: My Hello World

Log student actions?  [True]: False

Done.  Type runestone build to build your project

At this point you will have the following files and folders:

mynewproject/

        _static/

        _sources    

        _templates  

        conf.py

        pavement.py

  • The _static folder is for things like images or javscript files.

  • The _sources folder is where you will put your own writing. To start with there are a couple of examples files for you.

  • The _templates folder is for styling. There is a default set of templates that match the runestone interactive look and feel. That is a good thing to start with. Once you become more familiar with the system you may want to customize the templates or even make your own.

  • The conf.py file is used by Sphinx, and contains information from some of the questions you answered when you initialized your project.

  • The pavement.py file is used for building and setting build parameters.

All of these files are important, and you should not delete any of them.

Next run runestone build This command will create a build/mynewproject folder with an index.html file in it. If you want you can now run runestone serve and then go to your browser and open up the following URL http://localhost:8000/index.html Yay! You have a webpage. Feel free to explore a bit to get an idea about some of the components you can use in your lab.

Writing your Own Lab

OK, lets edit _sources/index.rst Initially it looks like this:

=====================

This Is A New Project

=====================





SECTION 1: Introduction

:::::::::::::::::::::::



Congratulations!   If you can see this file you have probably successfully run the ``runestone init`` command.  If you are looking at this as a source file you should now run ``runestone build``  to generate html files.   Once you have run the build command you can run ``runestone serve`` and then view this in your browser at ``http://localhost:8000``



This is just a sample of what you can do.  The index.rst file is the table of contents for your entire project.  You can put all of your writing in the index, or as you will see in the following section you can include additional rst files.  those files may even be in subdirectories that you can reference using a relative path.



The overview section, which follows is an ideal section to look at both online and at the source.  It is pretty easy to see how to write using any of the interactive features just by looking at the examples in ``overview.rst``





SECTION 2: An Overview of the extensions

::::::::::::::::::::::::::::::::::::::::



.. toctree::

   :maxdepth: 2



   overview.rst





SECTION 2: Add more stuff here

::::::::::::::::::::::::::::::



You can add more stuff here.

If you are not familiar with markup languages, this file should still be quite readable to you, and you can probably easily guess what most things do. Runestone uses a markup language called restructuredText. There is a very nice, short tutorial here.

To give you an idea of what you see in the example above, the section that starts with .. toctree:: is called a directive and it creates a table of contents for you. the maxdepth part sets the table of contents to show sections and subsections. And the line with overview.rst indicates that it is a file that should be included in the overall web page. More on all of this later. Our first task is simply going to be to wipe everything out, and start over. Using a plain text editor change index.rst to look like this:

=============

My Sample Lab

=============



Part 1: Turtle Graphics

=======================



In this section we will do the following:



* Create a turtle

* Make the turtle draw a box



.. activecode:: turtle1



   import turtle



   timmy = turtle.Turtle()

   for i in range(4):

       timmy.forward(100)

       timmy.right(90)





Now it is your turn.  Can you modify the program to make timmy draw an octagon instead of a square?

Now save the file and rerun the runestone build command. Everything should build without a problem and you can now run runestone serve and open up http://localhost:8000 from your browser. Notice that you can change the program and rerun it right from your browser.

It is probably obvious that you can create headings and subheadings. Unordered lists are created using * and the runnable code examples are created by the .. activecode:: directive. The name turtle1 must be unique on the webpage, other than that it is not used for too much at this point. The rest of the activecode directive contains plain old python code, but it must be indented to line up with the a in activecode. All indented lines are included as the body of the activecode directive, regular text processing starts at the first unindented line.

There you have it. You have created a very nice little lesson without a lot of hassle. The Runestone and Sphinx tools take care of all of the formatting for you!

Giving Students Browser Access to the Lab

If you have your own webpage hosted on a school server that you normally use for class you can make your Lab available to the students by simply taking the folder mynewproject inside the build folder and putting that on your website. The folder is self contained and can be hosted on any web server.

If you know the IP Address of your own computer and you simply want to give let students bring up the webpage from your computer you can do that too. For example, lets suppose you know that your IP address is 10.0.0.23 Your students can get everything they need from http://10.0.0.23:8000/index.html

Coming Soon

There are many free web hosting solutions out there and you can also choose one of them and upload your project folder for hosting there. I’ll cover at least one of them in another tutorial. In fact I think I see a whole series of tutorials in the future on topics such as:

  • Making an online quiz for class

  • Making a lecture or presentation

  • Hosting your lab or quiz on github pages or another online service

  • Using your lab with runestone services

easy publishing with runestone interactive

During my January travels, I also converted this blog from tubmlr, which had been frustrating me for a while, to Octopress, with which I have been very happy. Nothing like hacker level control of your own blog. But more, than just the switch in tools, the move to Octopress inspired me to make it easier for people to publish small or large works using the Runestone tools.

Yesterday, at the Learning @ Scale conference we demoed this new capability. See [the demo here](<http://runestoneinteractive.org/LearningAtScale). To make it super easy to publish:

  • Lecture slides

  • Demonstrations

  • A Tutorial

  • Lab Instructions

  • In class exercises

  • A short module on your favorite topic not covered elsewhere

  • An entire book

Building

You can simply follow the instructions at this new repository: github.com/RunestoneInteractive/RunestoneTools. In a nutshell:

  1. Install Sphinx, paver, and paverutils using pip.

  2. Clone the repository

  3. Edit the index.rst file in source, and add any additional rst files you may want, depending on how complex your project is.

  4. run paver build

Deploying

Now you have a choice. In the build directory you have a nice self contained set of html files, these files are set up to make use of the runestone server invisibly in the background. The static html can be served from any web server. Just drop in the build directory and you are ready to serve. OR, you can now host and deploy your project using GitHub Pages. To host on github pages you need to do three things.

  1. Create an empty repository in your github account.

  2. run paver setup_github_pages and paste in the URL of the new account.

  3. run paver deploy

Now your pages will be available at: http://youraccount.github.io/YourRepo

If you want to host these pages behind a custom domain name, you can follow the instructions on github for doing so. Hint: Its really easy.

I hope this new capability will inspire lots of people to give these tools a try. I also hope that we can build a repository of resources built with the tools, so that we can all share our teaching ideas. Stay tuned for more on that.

Caveats

All of the features, activecode, codelens, assessment questions, parson’s problems, and more work just fine. The major thing that will not work (yet!) is the login/logout. I need to rework our authentication system in order for this to work. This will for sure need to happen before the end of summer.

runestone interactive announces new editions of interactive textbooks

Today I'm really pleased to announce that  have launched version 2.0 of our interactive computer science  textbooks:

We first launched these books on our interactivepython.org website in May of 2012, after around a year of private testing in the classroom.  Since then we have had 1.3 million page views by a quarter of a million unique visitors.  Daily, we get around 2,000 unique visitors.  Not bad for a site with zero dollars for an advertising budget.

What Makes these Books Unique?

These books are unique because they are interactive.  We have developed a set of authoring tools that make it really easy to write an interactive textbook with many interactive features.  We call these the Runestone tools.  Some of the interactive features that are possible include:

  • Activecode:  Using a Javascript implementation of Python you can run and modify the examples in the textbook right in the book.  No server connection is required since it is based on javascript and runs right in the browser.
  • Codelens:  Using the amazing power of the pythontutor.com  tools you can step through examples one line at a time, forward and backward.  While you are stepping through the code you can see variables and other data structures change values.
  • Parsons Problems:  For beginning programmers Parson's problems are like refrigerator magnet poetry.  You can provide your students with the statements needed to write a program, but they must put the statements in the correct order.
  • Inline Quizzes: Each section of the book contains some inline quizzes that allow students to check their understanding of the material.  The quizzes have different feedback for each correct or incorrect answer that try to point students in the right direction. 
  • Online Homework:   At the end of each chapter are programming assignments.  In this new edition we have provided the answers to the odd numbered questions, and discussion forums for students to exchange ideas or ask questions about the homework problems.  As an instructor, you can grade your students programs on one convenient page.
  • Highlighting  This is another much requested new feature.  Students can highlight text using the mouse and the highlights magically reappear on any supported browser.  In addition we will remember the students last location in the book and offer to return them to that position when they return.
  • There are many other features but the best way to understand what we are doing is to actually have a look at our overview page, which shows everything I have mentioned here and a lot more in action.

Over this past year we have discovered that we serve two different audiences with these books.

  • Instructors looking for a textbook to use in their own course
  • People who are interested in teaching themselves some computer science and have found our books through google search, the Python wiki, or some other word of mouth source.

Textbooks as a Service

When we launched the site last year we decided to not only provide the books free and open for anyone who wanted to read them, but also as a service for instructors who wanted to have their own custom copy of the book where they could track their students progress, review their answers to quizzes, and grade their students homework.   If you want to use our books in your class you are welcome to do so.  You have two options:

  1. You can use a copy of either book as is with the order of the chapters just as they are on the books linked to above.
  2. You can try our custom interface where you can mix and match chapters from both books to create your own custom textbook.

Once you have created your own course then you will be able to see the assignments your students have completed right in the textbook.  I find this to be very valuable as an instructor.  For example if I have assigned the students to read and do the quizzes for a particular section, I can simply go to the quiz question and click on the 'Compare Me' button.  As an instructor I will see a summary of the answers my students gave, as well as the details of the answers that each student tried.

Supporting the Independent Learner

Perhaps the biggest surprise of this project is the number of people that have found one of the books through google, and are simply teaching themselves to program.  We are hopeful that some of the new features we have added will help foster a  community of learners so that people just learning to program can talk to others in the same situation.  Some things we hope are particularly helpful include:

  • Answers to odd numbered questions.  This was probably the number one request I got through email all last year.  How do I know if I did it right?  We decided to risk it and provide the answers, but only to the odd numbered problems.  In addition a student must try to answer the problem at least once before the answer becomes "unlocked"
  • Discussion threads for homework problems.  Again this may seem like a risky move where students can just publish their answer and others can copy.  But, what we are hoping for is that students will see that there are many ways to get to the "right answer"  There are different approaches and programming styles that can be used to solve the same problem.
  • Compare Me  Although we aren't sure about the title on the button, the idea is that after answering one of the quiz questions a learner can check on their overall 'grade' for all quiz questions, and see how their answer compared to all the other learners.  We haven't gone so far as to give out badges, but we think this is a nice intermediate approach.

The Runestone Tools

The books above were built using our Runestone Interactive toolkit.  These tools are freely available on github.  If you want to write your own interactive book, or even just use the tools to create some interactive labs for your students you are welcome to do so.  You can write your materials in an easy to use markup language called restructuredText and add examples or quizzes using very simple tags.  Complete documentation for our extensions to restructuredText is provided on the website.  In addition to our own books, the team at Harvey Mudd College has published CS for All  another introductory textbook using our tools.  I know of at least two other books in progress!  

If you are interested in following our development or getting involved You can do so in several ways:

Acknowledgements

I am grateful to the many people who have provided us with feedback over the last year.  And I am especially grateful to the ACM SIGCSE social projects committee for providing me with a special projects grant that allowed me to work with a student (Isaac Dontje Lindell) this summer.  He did a ton of work and will be graduating next year.  You should hire him.  In addition this project relies on many open source components which I will mention and link to below.

  • The original text for How to Think Like a Computer Scientist comes from Allen Downey, Jeff Elkner and Chris Meyers.  We have modified it a lot, but without a starting point for us to experiment with our interactive ideas this project never would have taken off.
  • The Problem Solving with Algorithms and Data Structures text is published as a paper textbook by Franklin Beedle and Associates.  Without the forward thinking of Jim Leisy this book would be stuck.  Thankfully Jim freed us to use the text in an interactive form online.
  • Mark Guzdial, Barbara Ericson and the rest of the CSLearning4U research group at Georgia Tech have provided questions, assessments,  and many other features and ideas.
  • The Activecode examples are made possible by skulpt
  • The Codelens examples are made possible by Philip Guo and his pythontutor.com
  • The look and feel of the book is based on the bootstrap templates
  • The system that builds the website from source is called Sphinx and is really the backbone of the system that allows us to write our interactive extensions.
  • Ville Karavirta wrote the original js-parsons library and Mike Hewner integrated it into the Runestone Tools.

making python3 my default

I'm finally there.  After a long time of writing books using Python 3, and teaching in Python 3 on a daily basis, I'm finally at the point where I'm changing my work environment to use Python 3 by default.

Here are the signs that convinced me it was time:

  • The release of Pillow to provide PIL functionality!
  • Django 1.5
  • IPython and IPython notebook are fully supported  -- I Love IPython Notebook
  • matplotlib !!
  • Sphinx, Jinja, SQLAlchemy, and many others are supported.  See: https://python3wos.appspot.com/

There are a couple of projects that I use a lot that are not yet on Python3 (web2py) but I'm not going to let that stop me.

It turns out to be pretty easy to get yourself up and running on all of this stuff with Python 3.  I'm on a Mac running 10.8.3.  The first and most important step is to get a working version of pip for Python3.  First you need to install distribute.

curl -O http://python-distribute.org/distribute_setup.py
sudo python3 distribute_setup.py

This will install easy_install in your Python home, but go one more step and install pip.

curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py
sudo python3 get-pip.py


This will create a pip in your /Library/Frameworks/Python.framework/Versions/3.3/bin  directory.

From there you can begin pip installing pretty much everything you need!

From my history today as I was setting everything up:

10393* pip3 install ipython
10409* pip3 install numpy
10410* pip3 install matplotlib
10429* pip3 install tornado
10430* pip3 install pyzmq
10455  pip3 install Sphinx

If you are on a Mac you will need to use easy_install to install one thing, for IPython.  For some reason pip installing the readline library puts it too late in the load path to work with IPython, so you need to use easy_install-3.3 readline  to get the history in IPython working just right.

Finally, you will want to add the Python 3 bin directory to your PATH.  Edit your .bashrc or .zshenv file.  Note that the following puts the Python 3 bin directory at the front of your PATH, making it the default when you type pip, python, or ipython.  If you need to revert back to python2.7 for some task you will need to be explicit about it.

export PATH=/Library/Frameworks/Python.framework/Versions/3.3/bin:$PATH

Easy!  You were expecting this to be a long and arduous process fraught with hacks and silly edits to config files.  Nope, just a few basic commands.  I take this as the final sign that Python 3 is here and ready to be your day to day Python.

to infinity and beyond

We passed 3 million entries in our log data for InteractivePython.org which got me to wondering about how we are growing. The site has been live for 275 days. You can really see how the site took off after day 100, which makes sense because that was bout the time fall classes started around the country. The tiny plateau between day 225 and 250 corresponds to Christmas/winter break for most schools, and now that Spring semester is in full swing it looks like the slope has gotten steeper again.

The dotted line is the linear best fit line with a slope of 12,836.7. Even if our log database is not growing exponentially thats an impressively steep slope.

And just because this is all about Python and education, here's the Python code that created the graph. I love matplotlib, it is such a powerful tool for quickly looking at your data.


import matplotlib.pyplot as plt

from numpy import polyfit, arange



f = open('bydate','r')

days = f.readlines()

totals = [0]



for d in days:

    day,count = d.split('|')

    count = int(count)

    totals.append(totals[-1]+count)



x = arange(len(totals))

m,b = polyfit(x,totals,1)

print m

plt.plot(x,totals,x,m*x+b, '--k')

plt.ylim(ymin=0)

plt.show()

The data file is simple. One day per line with the date in one column and the number of actions in the other column. Here are the last few days:


 2013-02-14 | 24349

 2013-02-15 | 17396

 2013-02-16 | 12645

everyday python - new blog

I've decided that it will be best to keep my personal blog separate from the new Everyday Python blog project. All of the infrastructure to make the posts interactive just work so much better when I have control of the server and everything else. So You can head over to Everyday Python and click on the Everyday Python link there, or if you want to subscribe to the RSS feed for that project you can use this link.

generating a password - part 1

OK, here is the first installment of the new Everyday Python series. Today I'm just providing a link, as I still have some infrastructure work to get done in order to publish each installment both here and at Runestone Interactive.

Generating a password part 1

introducing everyday python

Everyday Python

It is always risky to make your New Years resolutions public, but this is one I’ve

been thinking about for a while now, and so I’m going to go ahead and impose a

measure of accountability on myself by proclaiming this publicly. This week, I’m

starting a series of educational blog posts here, and as a part of my Runestone

Interactive publishing project.

My idea is to publish a series of Python programming projects, aimed at solving

everyday problems, or puzzles. Hopefully these little projects will be interesting

and will give learners motivation to dig into the details of the solutions. I

will provide links to the relevant Python programming constructs and data types in

either of the two interactive books (How to think like a computer scientist:

Interactive Edition , or

Problem Solving with Algorithms and Data Structures using Python) This will let me

focus on the problems, and not the minutia of the language. Lets face it, reading

about all the different possible string methods is not that exciting, but seeing

them in action, and then wondering about what else you might be able to do with

strings is OK.

The great thing about Python is that if you write things in a straightforward

manner its pretty easy to follow even if you don’t know all the details. So, that

is my intention. Write a solution and do it in straightforward Python that

beginners can understand. Each project will appear over several days, and

will likely include some homework related to the project. I’ll provide the solution

in a followup post. In addition, I may refine the solution over the course of

several days introducing more and interesting solutions or more advanced features

of the Python language.

In the back of my mind I am drawing inspiration from the old Communications of the ACM column by John

Bently called Programming Pearls In John’s columns he would feature a particular

problem or algorithm, and present it in a straight forward way, but then he would

refine that solution again and again polishing it until he had an incredibly

elegant solution. It was beautiful because even a novice programmer could

understand what was going on at the start of the article, but would get sucked in

to the beautiful solution and would learn more than they thought possible. While

advanced programmers might chuckle at the initial solutions, even they would have

to admit that they learned something by the end of each column.

I already have a few ideas in mind for the first few projects. I’m going to try

to start fairly easy with some string and list kind of projects, and work my way up

to more complex problems and algorithms, again this is meant to be educational so

that in theory a beginner might read through these posts more or less

chronologically, and learn some computer science along the way.

Click the title to leave a comment.

how to think like a computer scientist interactive edition

After hosting the interactive edition of How to Think like a Computer Scientist on the Google app engine at thinkcspy.appspot.com for over a year, we finally made the switchover to a new domain.  As of today all requests to thinkcspy will be automatically redirected to http://interactivepython.org

This new domain, is hosted by Webfaction.com.  They have very reasonable hosting rates, along with plenty of bandwidth and disk space.   I have more control over the development platform that I use, I get to keep my data in a real relational database (postgresql) where I can write queries and export my data as much as I want, and finally, if I ever need to move from Webfaction, I own the domain and can move it with me, and I won't need to go through this name change again.

For those of you who have used the thinkcspy site, you will notice a few upgrades to the content of the book have taken place.  In addition, we are now hosting another book along side thinkcspy.  So we now have coverage for both CS1 and CS2.

CS1 -- http://interactivepython.org/courselib/static/thinkcspy/index.html  

CS2 -- http://interactivepython.org/courselib/static/pythonds/index.html  

The CS2 book is based on our paperback book, Problem Solving with Algorithms and Data Structures using Python by Brad Miller and David Ranum.  Our forward thinking publisher has given us permission to convert the paper book into an interactive version and make it available online!

The other great thing about the move to interactivepython.org is that you can now host your own course.  That is you can use one of our books online, but you are the instructor, so you can grade homework assignments, and look at activity reports for the students in your class.  So far we have 20 different institutions hosting one of the two books for a course currently,  or for the upcoming fall term.  The links to create your own course are right there on the home page, so if you are interested in using this site for a course go right ahead.

The one downside to this move is that I did not migrate any of the data from the old thinkcspy site.  So if you have saved programs there you will need to recreate them at the new site.  I gave people plenty of notice about the move, so I'm hoping this won't turn into a big issue.  If it is, let me know, and I'll try to pull your code out of the google datastore for you.

The second issue with the move is that I don't use google accounts  anymore.  If you want to save your programs you will need to create an account on the site.

If you have a link to thinkcspy.appspot.com on any pages you control, please make the change to point to the new domain and location.  Thanks!

If you are interested in contributing to the project, have a book you would like to host, or are just interested in how we are doing this, you can check out the code on github:  [github.com/bnmnetp/r...](http://github.com/bnmnetp/runestone)   If you don't want to go that far, but have assignments, or assessment questions you would like to contribute, please contact me!

 

runestone interactive

What a week its been!   So much has happened in the last week that I just have to take some time out to organize my thoughts and get some of this information out there where others can read it.

What is Runestone Interactive?

Runestone Interactive is primarily a home for both the content and the technologies that go into publishing interactive textbooks.  At the moment interactive textbooks for Computer Science, but who knows how far this could go.  Probably the best way to see this is to actually visit our site and play around.  But in the meantime here is a short video to give you an introduction.

 

  1. Home to our open source publishing tools.  My goal as that these tools become the LaTeX of interactive publishing. There is lots of work to do before we can claim success here, but I think we have a good vision and a decent start.
  2. A model for a new online only publishing house. At some point, I think it is important for content to go through an editorial process, not just for copy editing quality, but for coherence, and quality of approach. Right now the number of content creators using our system is pretty small, but I can already see that starting to grow. We have lots of thinking to do about this.
  3. A home for hosting online courses or textbooks.

Who is your audience?

At the moment, I think we serve three distinct groups of people.

  1. The  teacher out there who wants a good quality textbook, but is not interested in forcing their students to pay $100+ and is interesting in trying an interactive book, all for free.   So to serve this audience we have interactvepython.org. You can select one of our pre-defined books and set up a course where you are the instructor and have access to grading etc. Or you can use the drag-and-drop interface to put together a book by selecting from the library of modules. Maybe some of these teachers have an exercise or two they would like to contribute, but mostly they are just looking for good materials to help them with their course.
  2. Authors who would like to contribute significant chunks of content -- chapters or section sized -- or videos. For example I'm working with an entrepreneur who wants to add content and a graphics package that looks more like Nodebox. He's very interested in putting together materials for middle school kids.  Others, including other Computer Science researchers, are more interested in algorithm visualization and working together on building in more visualization tools to the tool box.  Others have their own CS course materials that they would like to get into an interactive form and make available to the world.
  3. People who are interested in learning CS on their own.  We get a ton of people coming to the site just to learn on their own. I had the most heartfelt letter from a guy the other day who found our site as he is trying to retool after being laid off for a couple of years. He said this is the first thing he's come across that really worked for him and helped him learn a bit of CS.  How does this compare with Udacity and Coursera?  Both of those companies are primarily video oriented, and both of those companies are running courses that are in some sense synchronous.  You have to move through the course on their schedule to get the full benefit.  Our books/courses can be done at your own pace, on your own time.
 

I would like to use this for my own class how do I get started?

Its easy, just follow these steps:

  1. Go to interactivepython.org and register yourself as a user.
  2. Click on Create a Custom Course link in the right sidebar.
  3. Select a short name for your course that uniquely identifies it among all the other courses.
  4. Type in a short description for your course
  5. Now you get to choose.  Do you want to use one of our standard configurations, either How to Think Like a Computer Scientist: Interactive Edition, or Problem Solving with Algorithms and Data Structures using Python  or do you want to build a custom book by choosing modules from both of the above?
  6. Click on the build button.  Wait a few minutes and you will have a page with a link to your new course.
Now you can send your students to interactivepython.org and simply tell them to register for your course.  
 
Why would you choose to build your own course when you could just use one of the open books?  When you create your own course you become the instructor and admin for that course.  You can grade your students assignments right online, and even access their homework in progress if they have questions.  You can also get a sense for the different activities they are choosing to use in the textbook.

How can I get involved?

There are lots of ways you can get involved:

  • We need readers who are willing to be critical and report bugs on everything from typos to activecode blocks that don't work exactly right.
  • We need web developers who are willing to work on the back end, developing reports for instructors and making sure our infrastructure is sound.
  • We need web developers who are interested in developing user facing features, everything from algorithm visualizations to interactive assessment tools like the multiple choice and fill in the blank questions you see already.
  • We need compiler geeks who can help fix bugs in our Javascript implementation of Python.
  • We need authors who want to add new and interesting content, either entire courses, or new modules that could be combined with the modules already there.

hosting a web2py app in the cloud

I consider myself pretty up to speed on web development.

I’ve spent the last two days trying to find a hosting solution for the data structures eBook…. I looked at too many of the following:

  • Google App Engine — I’m a long time GAE user, but the setup I have uses sphinx to generate content on the fly which it wants to store in pickle files on the local file system.  This doesn’t work with a GAE and some other systems that treat the file system as read only and expect any writing you do to go to a database.
  • heroku — seems very promising, but I’m not sure how to configure web2py so it doesn’t need any local file system write access.  The lack of ssh access is a bit disconcerting.
  • dotcloud — seems promising but the app kept hanging for no discernible reason.  I fought with it for the better part of the afternoon before deciding to move on.
  • fluxflex — easy to get web2py installed, (1 minute)  but a real pain to try to work with a real app.  I’m not going to use the web interface for all my editing.  I want to git push and/or git pull to deploy and update the app.  It doesn’t seem possible to get both ease of installation and robust configuration management.
  • pythonanywhere - non-starter
  • webfaction — Its not free, but at least it has a command line I can work with and I’ve got the whole thing running.  The only confusing thing about webfaction is that they don’t list web2py as a supported framework, but when you type it in under other it magically works for  you….  The other annoying thing that I just discovered is that the 256MB limit is way too low for the default configuration.  I just now got a helpful email saying a support ticket had been opened for me because I was using too much memory.  This was like 15 minutes after I finally got my app running and I’d made about 5 page requests.

adding a module to skulpt

Now that I’m back from traveling in South America I’ve started on a project to add a turtle graphics module to skulpt.  Skulpt is a javascript implementation of Python written by Scott Graham, and is available on skulpt.googlecode.com.  Its a very nice project because it allows you to embed a Python interpreter right in your web page.  I’ve written about how to do this in a previous post.
The skulpt interpreter works like a charm, but it does not yet have any of the Python standard library modules implemented.  Scott has been working with wrapping some of Googles closure libraries, and has a webgl module too.  But none of your regular Python friends are available, things like math, random, turtle, etc need to be implemented.  For the most part these things are already available in javascript so the work is in creating the module and then wrapping the underlying Javascript inside the standard Python API.
Where to start?  There’s not a lot of documentation provided for skulpt, so I’m hoping this post will be enough to help others get going a bit quicker.  Scott was very helpful in responding to all my emails so this is not a criticism of him, rather I’m hoping this will save him the effort the next time someone wants to extend skulpt.  So, here’s the deal.  skulpt relies on two javascript files the first is skulpt.js  and builtin.js  A very minimal installation only uses skulpt.js, whereas if you want to use any modules they are in builtin.js.  Looking around the distribution you will not immediately find skulpt.js because you need to build it. You get a sculpt.js file by using the m script that comes with the distribution.  running m –help will give you the full list of commands, but the two that you probably most care about are m dist and m docbi The dist command builds both skulpt.js and builtin.js  docbi builds builtin.js and puts a new copy of it in the doc/static directory.
Lets begin with a quick tour of the source tree:

  • src - contains the implementation of the Python interpreter
  • src/lib - has the module implementations of webgl and goog.  This is where turtle will live and any other modules I implement along the way.
  • doc - This directory contains a google app engine application and is what you see on skulpt.org There are a couple of important files to check out in here.  One of them is doc/static/env/editor.js  This is the code that ties together the interactive editor on the home page with the skulpt interpreter and the codemirror editor.  If you know how to build a google app engine app then this directory makes sense.  One thing about the home page is that it is not set up to use any of the modules.  The modules are used in the more advanced ide, which you can find in doc/ide/static.  I’m going to tell you how to add modules to the simpler editor later in this article.
  • test - this directory contains a bunch of files for testing the implementation in a batch mode.  These tests are run whenever you run m dist, or m test.
  • dist - This directory gets created and populated when you run the m dist command.  It contains the built and compressed versions of skulpt.js and builtin.js
To illustrate how to make use of modules, here’s an extended version of my earlier hello world style example.

<html>
<head>
<script src=“skulpt.js” type=“text/javascript”></script>
<script src=“builtin.js” type=“text/javascript”></script>

</head>

<body>
<script type=“text/javascript”>
function outf(text) {
   var mypre = document.getElementById(“output”);
   mypre.innerHTML = mypre.innerHTML + text;
}

function builtinRead(x)
{
    if (Sk.builtinFiles === undefined || Sk.builtinFiles[“files”][x] === undefined)
        throw “File not found: ‘” + x + “'";
    return Sk.builtinFiles[“files”][x];
}

function runit() {
   var prog = document.getElementById(“yourcode”).value;
   var mypre = document.getElementById(“output”);
   mypre.innerHTML = ‘';
   Sk.configure({output:outf,
           read: builtinRead
              });
   try {
      Sk.importMainWithBody("<stdin>",false,prog);
   } catch (e) {
      alert(e);
   }
}
</script>
<h3>Try This</h3>
<form>
<textarea edit_id=“eta_5” id=“yourcode”>
print “Hello World”
</textarea>
<button onclick=“runit()” type=“button”>Run</button>
</form>

<pre id=“output”></pre>

</body>
</html>

There are some important differences between this version, and the version and the non-module version.  First off, the call to Sk.configure contains another key value pair which sets up a specialized read function.  This is the function that is responsible for returning your module out of the large array of files that are contained in the builtin.js file.  You will see that all of the modules are contained in this one file, stored in a big JSON structure.  The extra key value pair is:
read: builtinRead
The read function is just for loading modules and is called when you do an import statement of some kind.  In this case the function accesses the variable builtinFiles which is created from the builtin.js file.  The other difference, of course, is that you have to include builtin.js in your html file.  Note that builtin.js must be included after skulpt.js
Now as far as the module itself goes, the easiest thing to do is to start your module in the src/lib directory.  This way it will automatically get built and included in builtin.js.  If you don’t put it there then you are going to have to modify the m script, specifically the docbi function in the m script to include your module.  Suppose that you want to have a module called bnm.test  Here’s what you have to do.  First, you need to make a bnm directory under lib.  In this directory you will need to have either init.py or init.js or bnm.js to stand in for the bnm module.  There doesn’t need to be anything in the file as long as it exists.  This is just like CPython by the way.  Then to make a test module you can either make a test directory and put all your javascript code in init.js or you can simply create a test.js file in the bnm directory.  Lets look at the test module.
var $builtinmodule = function(name)
{
    var mod = {};
    var myfact = function(n) {
 if(n < 1) {
     return 1;
 } else {
     return n * myfact(n-1);
 }
    }
    mod.fact = new Sk.builtin.func(function(a) {
 return myfact(a);
    });

    mod.Stack = Sk.misceval.buildClass(mod, function($gbl, $loc) {
 $loc.init = new Sk.builtin.func(function(self) {
     self.stack = [];
 });
 
 $loc.push = new Sk.builtin.func(function(self,x) {
     self.stack.push(x);
 });
 $loc.pop = new Sk.builtin.func(function(self) {
     return self.stack.pop();
 });
    },
    ‘Stack’, []);


    return mod;
}

All modules start out with the $var builtinmodule = statement.
This test module exposes a single method to the outside world, called fact, There are a couple of key functions for building up a module.  The Sk.builtin.func   call for adding functions to your module, and the Sk.misceval.buildClass method.  This test module defines a simple factorial function called fact, and a class called stack.  Here’s a simple Python program that exercises the module:
import bnm.test
print ‘starting’
print bnm.test.fact(10)
x = bnm.test.Stack()
x.push(1)
x.push(2)
print x.pop()
print ‘done’

Its not obvious, but the buildClass method takes four parameters:  globals, func, name, bases
It seems that you always pass the mod object itself as the globals parameter, the func parameter is a function that represents the class object, the Name is the external name of the class, and bases presumably would be if the class is inheriting from another class.
The Sk.builtin.func method creates a function.  For module creation we typically only have to worry about the one parameter, func, which is the javascript implementation of our Python function.  The method can also take a globals object and two closure objects.  Look at the comments in function.js if you want more explanation of how the builtin.func method works.
Well, I think this should be enough to get you going.  Its worth repeating, if you made it this far, don’t forget to call m docbi or m dist after you make changes in your module, its easy to get into the mode of thinking that the new javascript is automatically loaded.  But builtin.js is not automatically rebuilt!
I’ll consider this post a work in progress, please leave a comment if something is unclear or you would like something explained in more detail.

python in your browser... in javascript!

I’ve recently been looking for ways to run Python in a browser. There are lots of reasons why I think this is a cool idea. For example, creating some simple tutorials, Making some online quizzes and tests, getting Python to run on my iPad…

It looks like there are two projects that are actively working on solving this problem. The first is called skulpt (see the demo on skulpt.org.) This project actually reimplements the core of Python in Javascript by hand. They are working on some modules, but right now it is basically the core Python language.

The other project uses Emscripten to compile the CPython C source code to javascript! Yep you read that right. You can see here that the core of Python 2.7 is running. You can import sys, but thats about it. Nevertheless both of these are really useful for beginning Python exercises.

The problem with both of these is that right now there is no user level documentation. Skulpt is particularly difficult as you can’t even look at the demo page to see how its done. You need to go digging through some additional source files from the distribution to figure out how to make it work for yourself. The source code of the syntensity page is pretty easy to follow. So here’s a hello world program using skulpt.

<script src=“skulpt.js” type=“text/javascript”>
</script>


<script type=“text/javascript”>
function outf(text) {
   var mypre = document.getElementById(“output”);
   mypre.innerHTML = mypre.innerHTML + text;
}

function runit() {
   var prog = document.getElementById(“yourcode”).value;
   var mypre = document.getElementById(“output”);
   mypre.innerHTML = ‘';
   Sk.configure({output:outf});
   try {
      eval(Sk.importMainWithBody("<stdin>",false,prog));
   } catch (e) {
      alert(e);
   }
}
</script>
<h3>Try This</h3>
<form>
<textarea edit_id=“eta_5” id=“yourcode”>
print “Hello World”
</textarea>
<button onclick=“runit()” type=“button”>Run</button>
</form>

<pre id=“output”></pre>


Its a bit on the verbose side, and it could certainly be improved so that less code would have to be replicated if you wanted to put multiple input/output areas on a single page, but as a learning example I think it works OK. The try/catch block around the Sk.importMainWithBody function call will capture Python runtime errors and throw them. This is a nice way to inform your users of syntax errors in any code they are trying out.

I’m certainly no expert in either implementation, I just offer this post as an example and a time saver for anyone else trying to get going with either implementation.

second edition of data structures book is out!

Now that I’m back from my JTerm travels, I’m happy to let you know that the second edition of Problem Solving with Algorithms and Data Structures using Python is available.  (www.fbeedle.com) David and I have worked hard on this new edition for about a year now, and we are really excited about it. We think it is a great step forward from the first edition, and I’m really looking forward to teaching out of it next year.


Here’s whats new:

  • All code is now written using Python 3.2
  • Chapter 1 now introduces Python sets, and exception processing.
  • Eliminated third party graphics package.  All graphics are done using the built-in turtle module
  • Focus on algorithm analysis in a newly written chapter 2.  In addition this chapter includes the analysis of Key Python data structures used throughout the book
  • New section on linked list implementation in Chapter 3
  • Moved Dynamic Programming to the end of chapter 4
  • Introduction of C style arrays and array management in Chapter 8
  • More focus on graphical recursive algorithms, including recursive tree drawing, and a recursive maze search program.
  • All source code for data structures has been organized into a single Python package to make it easy to incorporate into homework assignments.
  • Source for complete examples from each chapter are included so you do not need to piece together code from each listing
  • A new improved version of binary search trees in chapter 6.
  • New secion on balanced binary trees (AVL trees) added to chapter 6.
The main site for resources for this book is at www.pythonworks.org

In the meantime, here’s the cover for the second edition

PythonDScover.jpg

google app engine service login

So I’m working on an app during my sabbatical that has an iPad component and an online Google App Engine component. The Google App Engine part is half web based and half web service based. Of course this means that the local client part has to be able to authenticate itself to the Google App Engine before it can communicate and do useful stuff. Finding good reliable examples of how to do this is surprisingly hard. For the Objective C code I’m working on I found a nice set of classes that do the trick for you here: On Github For Python I found some example code on stackoverflow. However it was not really in a reusable form.

The basic outline of what you have to do is as follows:

1. Login to https://www.google.com/accounts/ClientLogin This will give you an auth token.
2. Use the token you gained in step 1 to login to your Google App engine application or service. When you have successfully logged in to your service google will set an ACSID cookie for you to use when you make subsequent requests to your service. This prevents you from having to login each time you make a web service request.

I’ve taken some ideas from both places mentioned above and have created a Python class for logging in and accessing app engine services from Python. To use this module you just need to import it and create a GoogleAppEngineLogin object. Once the object is created you can use the open method on the object to access further services. The open method is just a convenience wrapper around urllib2.urlopen but it also makes sure that your cookie has not expired before it makes a request. If you have comments or suggestions for how to improve the code please let me know via email or leave a comment.

The code is reproduced below, but you can also just download the file from git clone git@gist.github.com:36b1c45ed39298178907.git


import getpass
import urllib
import urllib2
import cookielib


class GoogleAppEngineLogin(object):
    “""
   Logging in to an App Engine Account (when you use google users) is
   a two step process: First you must login to Google generally.  This
   gets you an auth token.  The auth token is used as part of a
   request to login to your app/service During the login process for
   your app/service the server sets a cookie with the name of ACSID,
   it is this cookie and its value that serves as the authentication
   token for your own service/app.  So, for future requests you need
   to give the server the cookie as part of your request.  Handling
   cookies can be a bit tricky if you haven’t had some experience with
   it but luckily Python’s cookielib module makes it all pretty
   automatic.

    This class takes care of the whole login process for you, and then
    gives you a simple helper to access the URLs for your service.
    The helper function makes sure the cookie is still valid and
    passes on the request along with the cookie.  Technically you
    would not even need to use the helper function, you could use
    urllib2 directly to access your service but this seems a bit
    neater to me.

    Some of this code was inspired by and lifted from an example on
    stackoverflow.com, but that was all in-line code my contribution
    is to add some error handling and encapsulate the whole thing
    inside a class to make it easier to include in my/your own
    programs.  Here’s a link to the original thread on stackoverflow
    http://stackoverflow.com/questions/101742/how-do-you-access-an-authenticated-google-app-engine-service-from-a-non-web-pyt

    “""
    
    def init(self, user_email, user_pw, uri, source):
        “""
        Create a Google App Engine Object.
        Arguments:
        - user_email:  your google username
        - user_pw: your google password
        - uri:  The url of your google app engine service
        - source: The unique name of your google app engine service
        “""
        self._user_email = user_email
        self._user_pw = user_pw
        self._uri = uri
        self._source = source
        self._authtoken = None
        self._auth_cookie = None
        
        if not self.google_client_login():
            raise RuntimeError(“Could not login to Google”)
        
        if not self.app_engine_login():
            raise RuntimeError(“Could not login to your application”)

        
    def google_client_login(self):
        #
        # get an AuthToken from Google accounts
        #
        auth_uri = ‘https://www.google.com/accounts/ClientLogin'
        authreq_data = urllib.urlencode({ “Email”:   self._user_email,
                                          “Passwd”:  self._user_pw,
                                          “service”: “ah”,
                                          “source”:  self._source,
                                          “accountType”: “HOSTED_OR_GOOGLE” })
        auth_req = urllib2.Request(auth_uri, data=authreq_data)
        try:
            auth_resp = urllib2.urlopen(auth_req)
            auth_resp_body = auth_resp.read()
        except:
            return False
        # auth response includes several fields - we’re interested in 
        #  the bit after Auth= 
        auth_resp_dict = dict(x.split("=")
                              for x in auth_resp_body.split("\n”) if x)
        try:
            self._authtoken = auth_resp_dict[“Auth”]
        except:
            return False

        return True

    def app_engine_login(self):
        #
        #  Get a cookie
        #  we use a cookie to authenticate with Google App Engine
        #  by registering a cookie handler here, this will automatically store the 
        #  cookie returned when we use urllib2 to open
        #  http://www.google.com/accounts/ClientLogin
        self._cookiejar = cookielib.LWPCookieJar()
        opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(self._cookiejar))
        urllib2.install_opener(opener)


        serv_args = {}
        serv_args[‘continue’] = self._uri
        serv_args[‘auth’]     = self._authtoken

        full_serv_uri = “%s/_ah/login?%s” % (self._uri,urllib.urlencode(serv_args))

        serv_req = urllib2.Request(full_serv_uri)
        serv_resp = urllib2.urlopen(serv_req)
        serv_resp_body = serv_resp.read()


        for i, c in enumerate(self._cookiejar):
            if c.name == ‘ACSID’:
                self._auth_cookie = c
                return True

        return False

    def open(self,url,data=None):
        “""
        url should be a properly encoded url ready to go.  data is
        optional and should be used to provide parameters to pass
        along with the URL when you want to use POST instead of GET.
        If you provide data it must be properly encoded just as if you
        were calling urlopen directly yourself.
        “""
        if self._auth_cookie.is_expired():
            if not self.google_client_login() or not self.app_engine_login():
                raise RuntimeError(“Cannot get proper authorization for this request”)
            
        serv_req = urllib2.Request(url,data)
        return urllib2.urlopen(serv_req)
        

if name == “main":
    user = raw_input(“User: “)
    pw = getpass.getpass(“Password: “)
    service_url = “http://myapp.appspot.com”
    service_name = “myapp”
    gae = GoogleAppEngineLogin(user,pw,service_url,service_name)
    h = gae.open(“http://myapp.appspot.com/my/service")
    print h.read()


beautifulsoup, bonktown and growl

Here’s a little script that combines two of my favorite pass times. Python programming and cycling. bonktown.com is a great site that has steep discounts on road cycling gear. They only sell one item at a time and they typically sell that item until it is gone. I’ve gotten some great deals on clothing and other stuff on that site. Bonktown helps you know whats currently for sale in a number of ways, including a nice dashboard widget that pops up a notifier when something new comes on sale. The problem is that over time I’ve started to ignore the growl notifiers for bonktown, because I’m not interested in lots of the stuff they sell.

So, I wrote this python script that allows me to look for the stuff I am interested in buying. It works by having a file of regular expressions that I use to search the item descriptions when something goes on sale at bonktown. If the item matches something I’m looking for then I get a Growl notification. If not then I don’t hear about it.

Here’s the code:


#!/usr/bin/env python2.6

import re
import urllib
from BeautifulSoup import BeautifulSoup
import Growl

name = “MyBonk”   # was BonkMe
notifications = [“search_hit”]
notifier = Growl.GrowlNotifier(name,notifications)
notifier.register()

# Read file of search Terms
myTerms = open("/Users/bmiller/lib/bonk_items.txt").readlines()


# Get the latest page
bt = urllib.urlopen("www.bonktown.com")

doc = BeautifulSoup(bt.read())

itemlist = doc.findAll(id=re.compile(“item_title”))
price = doc.findAll(id=re.compile(“price”))
desc = doc.findAll(id=re.compile(“item_description”))

for term in myTerms:
    for i in range(len(itemlist)):
        if itemlist[i] and re.search(term[:-1],itemlist[i].contents[0],re.IGNORECASE):
            notifier.notify(“search_hit”,
                            itemlist[i].contents[0],
                            desc[i].contents[7].contents[0],
                            sticky=False)



This script makes use of several modules:

  • Growl

  • BeautifulSoup

  • urllib

  • re



I would have liked to use one of the standard library html/xml parsers, but I could not find one that was as convenient or easy to use as BeautifulSoup. If you can tell me how to parse messy html with one of the standard library xml modules please let me know.

mit uses python instead of scheme for intro cs

I’ve seen several interesting blog posts related to MIT switching from Scheme to Python for their intro course. This is really interesting to me since SICP is one of my favorite CS books of all time. Nevertheless I’m glad to see leading institutions change to my favorite language.

Paraphrase of Gerald Sussman’s remarks

Dan Weinreb’s blog

vpython version 5

The other day I discovered that the folks working on VPython have been very busy over the last couple of years. I had given up on VPython because the Mac distribution was so difficult to use. It required the X windows server and one point and lots of other extra junk at other times. But now things are easy! There is a great installer package and it just works.

For those of you who have not heard of VPython before it is a 3D graphics module called visual built on top of Python and OpenGL. You can do a lot of nice 3D graphics very simply with VPython. Its great for education in math and physics and introductory computer science.

So as an experiment I built a turtle graphics module on top of VPython. I use turtle graphics a lot in teaching but the TKinter turtles all suffer from some event loop problems when you use them with IDLE. VPython does not suffer from this problem plus gives you a lot of other cool benefits. One of the benefits is that the window resizes automatically for you based on the units you use in your application. Want to draw a picture at the atomic level of detail? No problem, want to draw planets circling each other? Again no problem.

Here’s a screen capture of a fractal tree created in 3-D using my new turtle.

Picture 2.png


Here’s the VPython turtle code:


def tree(t,trunkLength):
    if trunkLength < 5:
        return
    else:
        turnDz = random.randint(20,40)
        turnDx = random.randint(20,40)
        trunkShort = random.randint(10,20)
        t.width(trunkLength/10.0)
        if trunkLength < 25:
            t.color(color.green)
        else:
            t.color((174/255.0,145/255.0,0))
        t.forward(trunkLength)
        # right trunk
        t.right(turnDz,ZAXIS)
        tree(t,trunkLength-trunkShort)
        # left trunk
        t.left(2turnDz,ZAXIS)
        tree(t,trunkLength-trunkShort)
        # front trunk
        t.right(turnDz,ZAXIS)
        t.right(turnDx,XAXIS)
        tree(t,trunkLength-trunkShort)
        # back trunk
        t.left(2turnDx,XAXIS)
        tree(t,trunkLength-trunkShort)
        # restore
        t.right(turnDx,XAXIS)
        t.backward(trunkLength)



If you want to check out the turtle module and play with it, you are welcome to do so: hg clone bitbucket.org/bnmnetp/v… Or send me mail.