Python Injection

Today's fun stupid code is written in Python.

I have found a few times recently where it was handy to either replace or inject code in specific C++ files, or even level files.  I have done this sort of thing in AWK in the past, but AWK is, well, awkward.

The basic pattern looks like this:

import tempfile

src = "yourfile.txt"

with tempfile.TemporaryFile("w+") as out:
 with open(src, "rU") as f:
  for line in f:
   if line.find('REPLACE ME') != -1:
    out.write('REPLACE WITH')
   else:
    out.write(line)
 
 #Beginning of temp file
 out.seek(0)
 
 #Copy temp file over source
 with open(src, "w") as f:
  for line in out:
   f.write(line)

This just opens a temporary file, scans every line of the source file and writes it to the temporary file, with possible modifications.  Then it jumps back to the beginning of the temporary file and overwrites the source file with the temporary file.

It's probably a good idea to have whatever you're modifying backed up somewhere

It may be easier to just read the entire file into a list of strings and write that back out, but now you know how to use temporary files.  :)

What you do at the 'REPLACE ME' line is up to you - check for a specific sub-string, or exact line.  Similarly for the 'REPLACE WITH' line - replace the line with generated code, insert a new line of code and put the 'REPLACE ME' marker back in as a comment, etc.
I have one version of this that copies a component template .h and .cpp file, replaces 'XXXXX' in the new files with the new component name, and injects the component registration code into the game initialization function.  Creating a new component used to be a chore, now it's easy!

refptr

I come across this programming pattern every once in a while:

1. In an object's constructor, allocate some data
2. When copying the object, copy the pointer to the allocated data, rather than duplicate all of the data
3. Delete the data in the object's destructor

Of course, this ends in disaster, because the first of these objects that gets destroyed deletes the data, leaving all of the others holding an invalid pointer.

Some types of smart pointer let you pass "ownership" of the data when copying an object by only letting one object hold the pointer at a time, but this is very limiting.  (auto_ptr)

An alternative is to have a smart pointer that holds the "master" allocation, and all others get a regular copy of the pointer.  This works, but doesn't really fix the above problem where there is no obvious master allocation.

One of the best generic solutions I have seen is to have an intermediate reference counting class that all of the objects point to, instead of having them point directly at the data.  This causes an extra level of indirection when you want to dereference a pointer, but it solves the above problem without any manager classes, ownership passing, etc.  (shared_ptr)

So, something like this:

template <typename T>
class refptrsentinel
{
 template <typename U> friend class refptr;

 refptrsentinel(T *obj) : ptr(obj), count(1) { }
 ~refptrsentinel() { delete ptr; }

 void acquire() { ++count; }
 void release() { if(--count == 0) delete this; }

 T *ptr;
 int count;
};

template <typename T>
class refptr
{
public:
 refptr() : sentinel(NULL) { }
 refptr(const refptr &rhs) { sentinel = rhs.sentinel; if(sentinel) sentinel->acquire(); }
 refptr(T *obj) { sentinel = obj ? new refptrsentinel<T>(obj) : NULL; }
 ~refptr() { if(sentinel) sentinel->release(); }

 refptr &operator =(const refptr &rhs)
 {
  if(sentinel)
   sentinel->release();
  sentinel = rhs.sentinel;
  if(sentinel)
   sentinel->acquire();
  return SELF;
 }

 operator bool() const { return sentinel != NULL; }
 bool operator !() const { return sentinel == NULL; }

 T &operator *() const { return sentinel->ptr; }
 T *operator ->() const { return sentinel->ptr; }

private:
 refptrsentinel<T> *sentinel;
};

Light 'em up

It's lighting week.

First off, Blender's new Cycles render engine is pretty spiffy.  Path tracing just makes lighting look so nice.  Plus it's more or less interactive, as it can do a very low-res pass fairly quickly as you're moving the camera around.

Even better, Blender 2.7 added texture baking, so all of that yummy lighting goodness can be used to make lightmaps and such.

So far, the pipeline to convert a Blender shader graph into a GPU vertex/fragment shader has worked really well.  Obviously there are some limitations - path tracing and triangle rasterization are just completely different ways of doing things - but I can build up a complex set of materials and more or less get the same thing in the game.

Here's a ship in Blender:

And in-engine:

Not exactly the same, but similar.  It looks a little washed out, some of that is the bloom effect not present in the render.

This is using lighting based on the skybox - spherical harmonics for diffuse, plus a biased mip map sample to get a blurred / rough specular reflection.  It looks pretty nice in motion.  The emissive materials help as well.

Self-illumination and ambient occlusion maps would help a bit.  It may also be worth generating point/directional lights for hot spots in the skybox to give some added definition to the normal maps.

Some day we'll be doing real-time path tracing (or something similar), and it's going to be awesome.  :)

Points of note:

• Using the shader instruction texCUBEbias(...) to  choose a "blurred" mip level was not working properly.  The documentation on this instruction is pretty vague, but I kept getting really strange results.  texCUBElod(...) seems to do what I want.
• The auto-generated mip maps for the cube map definitely have seams at lower resolutions.  This seems a little tricky to fix, I may just live with it for now.
• Using D3DXSHProjectCubeMap(...) to generate the spherical harmonic coefficients was not working as expected - it looks like x and y are both negated?  Anyway, negating coefficients where the sign differences don't cancel out gives light coming from the right direction.

That's all for now, have a great week!

Getting the Message

Hello long-neglected blog viewers!

I won't get into a lengthy discussion about why I haven't posted for ages, but it sounds a lot like, "I can't post that because it's spoilers, I can't post that because it's boring, I can't post that because I'm busy working, I can't post that because I'm not at my computer, and I can't post THAT because... well, that raid's not going to heal itself now, is it?"

The short version: Game is still in progress. Engine is kind of nice. Music is coming along. Level design is not there yet, but making definite progress. I want to show stuff, but greyblock art is maybe not a good first impression, so I'm working on some prettying up.

Short-ish term goal:  Get a single level together for design and engine testing.

So, let's talk code.

typedef void (*Fn)(const string &);
typedef VSTACK<Fn> FnList;
typedef MAP<string, FnList> StringFnMap;

class cMessageHandler
{
 FnMap mFnMap;

public:
 void Register(const string &name, Fn fn)
 {
  mFnMap[name].push_back_resize(fn);
 }

 void Unregister(const string &name, Fn fn)
 {
  mFnMap[name].find_erase(fn);
 }

 void Call(const string &name, const string &s)
 {
  FnList &f = mFnMap[name];
  for(FnList::ITER it = f.begin(), end = f.end(); it != end; ++it)
  {
   (*it)(s);
  }
 }
};

Basically, it's just a list of functions listening to a particular message ID (string), and a function to broadcast a message (string) to each of them.  It's using my own containers, but you could use std::vector and std::map easily enough instead.

I also have a version that handles member functions - you just need to store the "this" pointer and a redirect function for each class, similar to a delegate.

This makes it pretty easy to send messages or triggers between components from data - there's no need to create a message ID or bind to specific functions in each class.  Instead of writing a bunch of custom code for each class to bind it to other components, they just need to send or listen to the same message name.

My project has a "game" library, which has many generic game components in it.  It is not supposed to have any game-specific code or logic in it to make it easier to re-use in a new project.  This was causing some problems connecting generic components to game-specific components, but this system lets me hook them up in data without the game library knowing anything about the game-specific stuff.

Initially I found this system kind of worrying, mostly because it involves so many strings.  It's the old-school hacker mentality, where individual bytes and cycles were of vital importance.  Well, you still probably need to be a little gentle with it - it's for events that happen only occasionally. Overall, I think it's worth being able to easily bind events from data.

So, yes, still working on things.  :)  I will try to update more often, not sure of the frequency yet!