You could trivially guarantee the polygon is always a hull by using a signed distance with Douglas-Peucker. You already know which side of each edge is “inside” the polygon. Ensuring a consistent winding order would allow you to calculate the signed distance from the candidate point to the simplified edge using a 2D “cross product” / determinant. From there, you can remove only vertices which are on the inside of the simplified edge.
Maxwelldoggums@reddit
You could trivially guarantee the polygon is always a hull by using a signed distance with Douglas-Peucker. You already know which side of each edge is “inside” the polygon. Ensuring a consistent winding order would allow you to calculate the signed distance from the candidate point to the simplified edge using a 2D “cross product” / determinant. From there, you can remove only vertices which are on the inside of the simplified edge.