Saying sorry is a sign of maturity. If you are with a person that never says they are sorry, it could be a sign they are immature.
And that can hurt a relationship. A person who never says sorry will sometimes ignore attempts to communicate by the other person and distance themselves. Then, after some time has passed, they will act like nothing is wrong. But that does not solve the situation, it just fosters resentment.
If this is you, work on saying sorry, even in situations that were not your fault. Is it more important to be right, or to be in a loving relationship? Show your significant other that you love them by giving up your right to be right.