Datafied societies need informed public debate about the implications of data science technologies. At present, internet users are often unaware of the potential consequences of disclosing personal data online and few citizens have the knowledge to participate in such debates. This paper argues that critical big data literacy efforts are one way to address this lack of knowledge. It draws on findings from a small qualitative investigation and discusses the effectiveness of online critical big data literacy tools. Through pre and post use testing, the short- and longer-term influence of these tools on people’s privacy attitudes and behavior was investigated. The study’s findings suggested that the tools tested had a predominantly positive initial effect, leading to improved critical big data literacy among most participants, which resulted in more privacy-sensitive attitudes and internet usage. When analyzing the tools’ longer-term influence, results were more mixed, with evidence suggesting for some that literacy effects of the tools were short-lived, while for others they led to more persistent and growing literacy. The findings confirm previous research noting the complexity of privacy attitudes and also find that resignation toward privacy is multi-faceted. Overall, this study reaffirms the importance of critical big data literacy and produces new findings about the value of interactive data literacy tools. These tools have been under-researched to date. This research shows that these tools could provide a relevant means to work toward empowering internet users, promoting a critical internet usage and, ideally, enabling more citizens to engage in public debates about changing data systems.