Stephanie, thank you for adopting Alfie and giving him a forever home despite his fears. It's people like you who make the work we do so worthwhile!
The first thing to do is relax. If you're stressed by the situation, Alfie will certainly pick up on your feelings and react accordingly. At this point, it's best to treat Alfie as if he were a feral cat. Let him set the pace for your interaction and he'll come out of his shell gradually over time.
We have a post on taming ferals. If you have a moment, please take a look at it here: http://kittyhelpdesk.com/help-desk/-taming-feral-cat
There are a couple of things you can do to help him along. The first is related to the chair you mentioned. In his six years at the shelter, Alfie put up a number of defenses to protect himself. One of those was the need to retreat and hide in a safe place with close walls. He saw the area under the chair as his safe zone until you moved the chair and he realized that it wasn't safe at all. You need to give him his own little spot where, when he's there, you never interact with him. It can be a small cat cube or just a box with some holes cut in it. You might just crack a closet door so he can go in and out. But whatever you do, when he's there, you should never try to touch him or move him. He needs his safe hidey hole in order to gain the confidence to come out. Just knowing that he can retreat there can make a big difference for him.
Be careful to only use your closed hand when offering contact to Alfie. To a human, a fist means the threat of violence, but to a cat, it looks more like a paw. An open hand looks more like a paw with claws extended and is much more threatening to a cat. Add to that the fact that Alfie was very likely handled a lot at the shelter and you end up with a cat who may actually have developed a fear of grabby human hands.
One exercise that we've found to be very effective is to sit on the floor of the room where Alfie is and read quietly to him. Let him get used to your voice and your presence while you're focused on the book. Don't look at him or reach out to him, even if he initiates contact. This is how cats in the wild indicate their trustworthiness. It may take many sessions, but Alfie should eventually reach out to you.
Stephanie, we hope you won't give up on Alfie. He has six years of confinement to overcome. It may take a long time, and he needs to take it at his own pace. He's still in a fight or flight panic mode based on his experience being confined for so long. Much like a human prisoner, the close confinement can actually become comforting.
It will always be two steps forward and one step back for Alfie until he finally overcomes his psychological hurdles. In order for you to be successful, you need to detach yourself from any particular outcome. If you're disappointed at Alfie's reaction, he will sense that. Try and be positive and take it as it comes.
After six years in jail, he developed tools that helped him to survive. Please don't confuse survival with thriving. Yes, he survived, but he will never thrive in a cage. He can eventually thrive in your home and under your care but it will take a long time. He knows that. he's just working from the toolkit he has and that toolkit was built from fear.
One additional thing that may help is to add another, very low-key and friendly cat to the mix. A second cat can often help assuage these situations by being a buffer for the frightened feline. A kitten may have too much energy for Alfie, but a young, laid-back cat could be just the thing. It just has to be the RIGHT cat. A domineering cat certainly wouldn't do him any good right now.
I encourage you to try and be nonchalant about his situation when he's hiding. Just go about your business and try your best to be optimistic, especially when you're around him. Believe me when I say that Alfie will pick up on that. These things often require a lot of time--sometimes a year or more for significant progress, and you've already seen some good progress from Alfie already. Relax and enjoy the process. With a lot of patience and a relaxed attitude, you can work wonders with Alfie. We wish both of you all the best!