The materials used to upgrade weapons and armor can be purchased from the blacksmiths that do the upgrading, however the price of the metals and jewels they sell is rather steep, and ultimately not of desirable quality.While adventuring, you may defeat monsters or find hidden treasure that contains chunks of metals, or rocks that can be cracked open to reveal jewel deposits inside. These materials can be of higher quality than what is sold by the blacksmiths, and will result in a better outcome for your upgraded gear.
Remember that any upgrades made to your equipment can be removed and replaced with higher quality material, so in some cases it may be beneficial to use what you have and worry about quality later.