As robotic arms are deployed in increasingly unstructured spaces and close to people, they have to satisfy new requirements in terms of path quality and environmental complexity. In particular, the motions of robotic arms must look natural to people, while they can also address difficult instances involving tight, cluttered spaces, such as objects stored in shelves and tightly arranged one next to each other. This work first proposes an informed sampling-based motion planner for picking objects in clutter, which provides anytime properties in terms of cost metrics that result in natural looking motion. The method is focused on achieving good computational performance in addressing hard problem instances. The second contribution deals with the case where multiple arms need to coordinate to solve manipulation challenges. Borrowing ideas from multi-robot planning, a scalable asymptotically optimal motion planning algorithm is proposed to effectively deal with problem instances that involve multiple arms. This method is extended to also address the coordination of bi-manual humanoid robots, where the two arms share degrees of freedom. These methods plan over an implicitly derived tensor product of individual arm roadmaps. The existing contributions are laying the foundations for progress in task planning for multi-arm manipulation that is both computationally efficient while also providing desirable asymptotic optimality guarantees in terms of appropriate cost metrics.