Here is one way to solve a difficult problem. It is described in the context of developing a software solution but the process can be similar for a broad selection of problem domains.
First, ask questions. Ask lots of questions. Ask every question that you can think of. Questions are more important than answers, especially at this stage. Do not be tempted to try to answer these questions at this point. If you look for answers too early, you may stop asking questions before you’ve thought of the important ones.
Write them down as you ask them. You’ll be surprised at how quickly you will forget them if you don’t write them down. Also, if you write them down you can read them later and evaluate them from a fresh perspective. Not only can you read them later, you should look over what you’ve written. See if you have forgotten anything. See if there are any patterns to be discerned among them.
At this point, you can start looking for answers. That doesn’t mean that you shouldn’t capture any good questions that occur to you while you do. Consult with people that are familiar with the problem. In the case of a software project that would include the intended users of the program.
Write a concise description of the problem as you understand it. Review the questions and any answers that you’ve found to see if you have overlooked any details in your problem description.
Next, imagine potential solutions. Write them down as you think of them. Frame them in the form of stories from the perspective of the user of the program. Try to think of several different approaches. Read what you have written and see if any of these stories can be broken down into smaller stories. Keep breaking big stories into collections of smaller stories until you feel like you could write a program that implements one of the small stories.
At some point, pick one of the small stories. You might pick an easy one. That will let you see results quickly and build your confidence. You might pick a hard story. You may have to struggle more to implement it but you will have a sense of accomplishment when you are done with it. After implementing each story you should write a test framework that demonstrates that it works.
This description has been written as a linear sequence but often in practice it unfolds iteratively. You start out asking questions. You think you are ready to look for answers to them but you think of more questions. The more you learn, the more questions that you have.
As you start imagining solutions your understanding of the problem may be clarified so you can revise the problem description. You may start to implement a story and decide that it should be broken into smaller stories. You may think of more questions at any stage. This is as it should be.
Don’t be afraid to start trying to implement a solution. There is such a thing as analysis paralysis. Software is cheap. The raw material for it is ideas. The principle cost is labor and that is relatively cheap in the broad scope of things. Do experiments along the way to help you understand the problem better. Experiments can also inspire story development.
Finally, understand that you will rarely find a problem that you will be able to completely solve. Usually the best you will be able to do is create a solution that is good enough. It remains for you to decide when you’ve achieved that stage.
This sounds simple but it is hard work. Just remember that you haven’t failed until you quit trying. Sometimes a good night sleep can inspire new perspectives on the problem. Sweet dreams, don’t forget to tell the people you love that you love them, and most important, be kind.