People love auctions -- auctions of all variety. A fastest growing area in the auction martet for the internet is "niche auctions."
Let's face it. As far as the large "all and everything" auctions go eBay and Amazon have it sowed up. Even Yahoo with all their money and advertising couldn't really break into the field.
So how can anyone expect to compete with the "big boys?" Well, you can't. So don't try. Period.
Rather than dreaming of going head to head with eBay or Amazon why not take advantage of what you know. And presumably that is the area defined by your website. A small niche auction targeted toward your specific interest group has a great chance of success.
Auction software is available. The cost of the software varies tremendously depending upone required features. But be forwarned. When you shop for auction software it will be a question of selecting the best package at the best price. Perfect has no place in this process. There will be something about each package you would rather be different. Just look for the best fit of functionality and cost that you can find and go for it.