For the activated sludge wastewater treatment process, the image segmentation of flocs and filaments has become a crucial component in the successful implementation of a sludge volume index (SVI) sensor and the early fault detection of filamentous bulking. The segmentation of a phase contrast microscopy (PCM) image is a challenging problem because of the weak greyscale distinction between flocs and filaments, as well as the artifacts of halos and shadows. In this work, we proposed an automatic floc and filament segmentation method for PCM images using a U-Net deep learning structure with data augmentation. A loss function combining the binary cross entropy (BCE) function and Dice coefficient is proposed to improve the segmentation accuracy and sensitivity with unbalanced foreground and background samples. The performance of the segmentation algorithm is evaluated by the accuracy, precision, recall, F-measure, and intersection-over-union (IoU) metrics. Lab-scale experiments on the activated sludge process have been carried out to verify the proposed image segmentation method. Our proposed U-Net models with the combined loss function give better results than the U-Net models with BCE, fully convolutional network-VGG16 (FCN-VGG16), and a traditional segmentation method.