The short answer is yes. You can use one of two methods to achieve the result.

First method:

Install the Images for Titles mod from Free Addons, following the instructions in its readme.
Upload an image file for each sidebox heading you want to substitute, named (for instance) boxhead-categories.gif in the case of the Categories sidebox.
Save the image in /includes/templates/CUSTOM/buttons/ENGLISH/.
If you want to use other than .gif files for the images, edit the define in/includes/languages/ENGLISH/extra_definitions/CUSTOM/image_titles_defines.php.

This method automatically removes the [more] that follows a replaced heading link; but it does require that the same image type (gif, jpeg, png) be used for all replaced heading images in an area.

Second method:

Using an image editor, create your image sets with a name for each sidebox you want to change. Upload to your server and save the images in /includes/templates/CUSTOM/buttons/ENGLISH/.

In your /includes/languages/CUSTOM/english.php find this line of code:

define('BOX_HEADING_CATEGORIES', 'Categories'); 

define('BOX_HEADING_CATEGORIES', zen_image(DIR_WS_TEMPLATE . 'buttons/' . $_SESSION['language'] . '/' . 'your_image.gif', 'Your alt & title text for heading')); 

define('BOX_HEADING_LINKS', '  [more]'); 

define('BOX_HEADING_LINKS', ''); 

Replacing background images for one or all sidebox headings:

You may want to keep the standard heading text, but change the background images.
If you want to replace the default Zen Cart background image to be used for all the sideboxes use an image editor to create your image and save it to
/includes/templates/CUSTOM/images/ and upload it to your server.

Now open your stylesheet.css, find and edit the following rules:

.leftBoxHeading { 
  margin: 0em; 
  background: url(../images/your_background_image.jpg) no-repeat left top #ff6699; 
  padding: 0.5em 0.2em; 
}
.rightBoxHeading { 
  margin: 0em; 
  background: url(../images/your_background_image.jpg) no-repeat left top #ff6699; 
  padding: 0.2em 0em; 
} 

#categoriesHeading { 
  margin: 0em; 
  background: url(../images/your_categories_background_image.jpg) no-repeat left top #ff6699; 
  padding: 0.2em 0em; 
}