You probably don't need to do it. GNU Smalltalk supports dynamic linking, and it's probably easier if you make a plug-in module instead. This is how most library bindings work.

If you are interested in how the virtual machine works, however, modifying it may be very instructive. The code is well commented.