We are combining symbolic and geometric planning to synthesize human-aware plans in order to deal with the complex and highly intricate planning problems induced by Human-Robot collaborative object manipulation. In this paper, we summarize our previous contributions - refining symbolic actions at geometric level, during the symbolic planning, in order to assess their feasibility and computing the geometric side effects-, then we present the current contributions meant to tighten the cooperation between the symbolic planning and the geometric planning: the symbolic planner helps the geometric one by providing it with constraints and domain-expert knowledge making the geometric planner more efficient, and the geometric planner helps the symbolic one to find the best plan based on social costs computed at geometric level. We also propose different examples, highlighting the interest of such cooperation between the planners in simulation and on our PR2 robot.