It was definitely about slavery. Not so much in existing states — at least not directly. What it boiled down to was whether future states admitted into the Union would be free or slave because they feared an imbalance would end up threatening the institution as a whole.
The South’s economy was slave based, so naturally that would have been their chief concern (just look at the post-war economic collapse), but culture definitely played a substantial role as well. The antebellum lifestyle was simply unsustainable after slavery had been abolished.
Where the question of self determination comes in is with regard to whether the states could legally secede, and even more to the point, whether the federal government had the authority to STOP them from leaving. And that’s a subject that continues to be debated to this day.