Here are a few resources to help in understanding why God may permit evil.
The very short answer here is that evil is a product of the mind that has the capacity to choice; ie. free will. God is love and as we are made in His image we cannot love (or do the good that love produces) without the ability to choose it. It would not be a choice if there was no alternative to love; ie. hate or the product of hate which is evil.
Can one choose to do good without believing in God? Yes and no. A belief in God may not motivate goodness. But the capacity for goodness is not possible unless one were created with that capacity initially.
Deliver us From Evil – ex-athiest
God is our refuge and strength,
an ever-present help in trouble.
~ Psalm 46:1