Why do you think Mr. Rochester disowns Adela and acts like she isn't his daughter?

When Mr. Rochester talks to Jane about his past and about Céline he talks how Céline is Adela's mother but he is not her father. He keeps his distance from her and doesn't pay much attention to her in the short times that he is home. I feel like Mr. Rochester wants to forget about his past and his wife leaving him, so he acts like Adela isn't his child. After Jane hears this from Mr. Rochester she proclaims that she will comfort Adela even more because Mr. Rochester won't do it himself. I'm not sure why Mr. Rochester would feel like Adela isn't his own, but I feel like there has to be something more in his past that has made him feel this way towards Adela.