Evaluate Adding Androidmeda Tool

[cpholguera]

See description and discussion in #3624

[In3tinct]

Hi @cpholguera

May i know what is the criteria for discussion to be approved and turned into an issue or a PR. I reviewed this https://mas.owasp.org/contributing/3_PRs_and_Reviews/#how-to-participate-in-discussions

There have been 2 interactions/discussions so far, which i have answered.

1. Add Androidmeda Tool #3624 (comment)

2. [Question] [API] provider base url In3tinct/Androidmeda#15

i'd also like to make a case that tool is already being used by others (313 stars and 39 forks so far). 🙂
Also, some blogs that covered it.

https://fuzzinglabs.com/llm-assisted-android-deobfuscation-benchmark/ (this is a great benchmarking)
https://www.mobile-hacker.com/2025/07/22/deobfuscating-android-apps-with-androidmeda-a-smarter-way-to-read-obfuscated-code/

Thank you.

[TheDauntless]

I took a look at the project but I don't immediately see the added value? As far as I can tell Androidmeda uses an external decompiler to dump files and submit them to an external LLM with an additional prompt. Is that correct? What's the advantage over decompiling with JADX and simply opening the decompiled output with vscode and asking Copilot to find vulnerabilities? What is the added value over something like JADX-MCP-server (which also currently isn't listed) but at least integrates directly with JADX-gui? Maybe I'm missing something?

Using non-deterministic tools for security assessments is also something I'm really not a fan of as it will give both false positives and false negatives which often can't even be reproduced. Currently all of the tools we have listed in the project are deterministic. We would need to update the site to frame such tools. Even though the target audience of the MAS project is technical, unfortunately so many people just don't understand what an LLM can and cannot do.

We have some requirements listed for adding tools (https://mas.owasp.org/contributing/#tool-inclusion-disclaimer-for-contributors) and from what I can tell so far:

• It's not open source (due to requirement of LLM).
• It doesn't provide unique value (vs https://github.com/craftysecurity/jadaix or https://github.com/pranavkorhale/APKSecureScan or https://github.com/aiwithVijayakumar/AnReview vs simply asking chatgpt to create such a script) and I fear if we allow this tool (which is just glue between jadx and an LLM) there is no reason to deny any of those other tools. The internet is already being overrun with basic AI-generated tools and opening this door will just add even more work on top of all the AI spam we're already facing in the PRs.
• It's not deterministic.

[In3tinct]

fair assessment, just one correction - i would consider it open source, since it also integrates with Ollama (local inference, modern day general CPUs are fine).

We may need to re-consider the non-deterministic criteria with LLM for tool addition in the future as more and more tools integrate with it, it does seem to be bring added value expanding what tools can do. my 2 cents. :)

[TheDauntless]

Sorry, I missed your comment. While I agree that the possibility to integrate with anything from Ollama makes it more open source, what I really mean is that we can't see / audit the actual code that is finding vulnerabilities. An LLM is considered 'closed source' since we basically have no idea what it's really doing. In this regard, there's no real difference between a tool that uses an LLM and one of the many online closed-source security scanners.

The tool still touches on a much bigger issue in regards to determinism and soundness. Personally, I feel like using LLMs to find security issues is something we could at some point add, but then mostly to explain the advantages and limitations of such a tool, without mentioning specific tools. On the other hand, using LLMs for RE and de-obfuscation works really well, since it is inherently a non-deterministic process (different reversers will use different flows / refactorings) and this is something that I feel we could add at some point. In both cases we would need an AI disclaimer.

Finally, (mostly for any feature readers), the providing unique value is even more important now, with AI. If your project can be recreated by AI in just a few prompts, it doesn't really add any value as it's probably faster and easier to have AI create a new script for whichever use case you're working on.