Productive Struggle With Programming
I recently came across an article online regarding converting a color image into a pencil-style “sketch” using Python code. The desktop version of the code uses a sophisticated Python package called cv2 (developed by Intel®) which is used for high-end applications like face detection, agricultural surface analysis, car license plate detection (toll-by-plate), and many other vision scenarios.
It was not hard to get the desktop application to work by pasting the code from the article into the Python editor on my TI-Nspire™ CX II graphing calculator. I even added code to show the progression behind the scenes.
From the screen above you can see that it’s a five-step process with the final step:
• divide the gray image by the inverted blurred image
The challenge for me was to code this entire process using Python on the TI-Nspire™ CX II graphing calculator which supports image processing using the ti_image module but lacks the power of the cv2 package. Spoiler alert: I was eventually successful … after productive struggle.
Many photo editing applications contain an arithmetic menu for adjusting the values of the individual pixels or the arithmetic is buried inside some of the editing tools. After all, an image is just a 3D matrix: rows, columns and RGB values.
To work with images on the TI-Nspire™ CX II graphing calculator, the properly sized image must reside on a Notes app and have a name. In the Python program, start with
from ti_image import * ;# found on the More Modules menu
orig=load_image(“>name”) # replace the name placeholder with the name of your image
orig.show_image(x, y) # start with (0,0) and adjust as appropriate
When you type the name of an image variable and a period (orig.), you get a pop-up list of tools available for that variable, such as show_image(x, y). For more practice with image processing on the TI-Nspire™ CX II graphing calculator using Python, see the modules section of TI Codes for some TI Image mini projects.
Steps 1 and 2. Convert the color image into a (1) grayscale and (2) negative image. Pretty easy. Just “equalize” the pixels so that they contain (approximately) equal amounts of red, green and blue. Then “invert” the value by subtracting it from 255.
|
|
Note: The commented line # m=int(0.2989*r+ 0.5870*g+ 0.1140*b) uses a weighted average of the three colors to give a better rendition in grayscale due to the human eye’s perception of the three colors.
Step 3. Blur the negative grayscale image. This was easier for me than it sounds. I contacted a friend and colleague, Fred Fotsch, and got a copy of an image-processing activity he wrote that included the blurring code (Gaussian blur) and adapted it to my needs.
blur.show_image(164,5)
Note: If you are interested in the convolve( ) function which contains several photographic “filters,” contact me.
Step 4. Create a positive image of the blurred negative. This is the same routine as creating a negative from a positive, so it could have been a function.
|
|
Step 5. I thought this would be the hardest part. I wrote a divide function that takes two images as arguments, examines each pixel and divides the value of each color channel by its companion in the second image. Since the image is grayscale, all three of a pixel’s color channels have equal value. I only needed to process one of the channels.
|
|
Note: Scale is used to expand the quotient to a value between 0 and 256.
But the resulting image seen above is not even close to what I expected! There’s nothing more frustrating than a program that runs but does not give the results I want. That means there’s something wrong with my methods, and the computer cannot tell me what it is.
Long story short: Combining Steps 1 and 2 was my big mistake. The published algorithm specifically stated to divide the Step 1 grayscale image by the Step 4 blurred grayscale image to get the sketch. It took a whole day and a lot of incorrect variables in the wrong places to get it right, but the result is worth the effort. When all else fails, read the directions!
Giving the project some further thought, I wondered why the two “negative” steps were needed at all; after all, creating a negative then a positive is just an inverse function that results in the original data, right? Consider the mathematics: f( f⁻1(x) ) = x. Reducing the algorithm from five steps to three steps saves some computing time and gives the same desired output.
Finally, I decided to eliminate the grayscale step completely and work directly with the color image: original - blur - divide. The result using just two steps is just as interesting with a “colored pencil” effect.
Try the image2sketch.tns document yourself. It contains all three versions of the algorithm. Feel free to change the original image but keep the screen dimensions in mind: 319x211. Your image should fit the left half of the screen, so a maximum size of 150x210 should be safe. My image is 134x200. A smaller image will let the programs run faster since there are fewer pixels to process.
It is really humbling to struggle with a problem and super satisfying when you finally reach the solution. Moral of the story: When you are sure everything is correct and your program still doesn’t work, then one of the things you are sure of is wrong. Enjoy the struggle!
About the author: John is a retired teacher splitting time between sailing in New Jersey and mountain biking in Florida (did he get that backwards?), still getting kicks out of working with TI to provide feedback on new products and developing meaningful programming content for mathematics and science … and still having fun with all the graphing calculators and the accompanying toys.
Photo credit: Image of child provided by J. Hanna and used with permission.
Intel® is a registered trademark of Intel Corporation or its subsidiaries in the United States and other countries.
