diff --git a/Packs/SilentPush/.secrets-ignore b/Packs/SilentPush/.secrets-ignore index 5c62e1699051..8c28e85b2eec 100644 --- a/Packs/SilentPush/.secrets-ignore +++ b/Packs/SilentPush/.secrets-ignore @@ -1,2 +1,2 @@ - https://api.silentpush.com +https://api.silentpush.com 440 \ No newline at end of file diff --git a/Packs/SilentPush/Integrations/SilentPush/README.md b/Packs/SilentPush/Integrations/SilentPush/README.md index 22e660856391..be46b25ba797 100644 --- a/Packs/SilentPush/Integrations/SilentPush/README.md +++ b/Packs/SilentPush/Integrations/SilentPush/README.md @@ -43,8 +43,11 @@ This command queries granular DNS/IP parameters (e.g., NS servers, MX servers, I | SilentPush.DensityLookup.records.nssrv | String | The name server \(NS\) for the query result. | #### Command example + ```!silentpush-density-lookup qtype="nssrv" query="vida.ns.cloudflare.com"``` + #### Context Example + ```json { "qtype": "nssrv", @@ -57,9 +60,11 @@ This command queries granular DNS/IP parameters (e.g., NS servers, MX servers, I ] } ``` + #### Human Readable Output >### Results + >| Field | Value | >|---------|------------------------------| >| Density | 100601 | @@ -113,9 +118,11 @@ This command performs a forward PADNS lookup using various filtering parameters. | SilentPush.PADNSLookup.records.type | String | The type of the DNS record \(e.g., NS\). | ### **Command Example** + ```!silentpush-forward-padns-lookup qtype="ns" qname="silentpush.com"``` ### **Context Example** + ```json { "qtype": "ns", @@ -136,6 +143,7 @@ This command performs a forward PADNS lookup using various filtering parameters. ### **Human Readable Output** >### Results + >| Field | Value | >|--------------|------------------------------| >| Answer | henry.ns.cloudflare.com | @@ -178,9 +186,11 @@ This command retrieve the reputation information for an IPv4. ### **Command Example** + ```!silentpush-get-asn-reputation asn="12345"``` ### **Context Example** + ```json { "asn": "12345", @@ -193,6 +203,7 @@ This command retrieve the reputation information for an IPv4. ### **Human Readable Output** >### Results + >| Field | Value | >|--------------|------------------------| >| ASN | 12345 | @@ -233,9 +244,11 @@ This command retrieve the takedown reputation information for an Autonomous Syst | SilentPush.ASNTakedownReputation.takedown_reputation.asn_takedown_reputation_explain.listings_max_age | Number | The maximum age \(in hours\) of the listings, indicating how recent the flagged IPs/domains are. | ### **Command Example** + ```!silentpush-get-asn-takedown-reputation asn="211298"``` ### **Context Example** + ```json { "asn": "211298", @@ -249,6 +262,7 @@ This command retrieve the takedown reputation information for an Autonomous Syst ### **Human Readable Output** >### Results + >| Field | Value | >|------------------------------|----------------------------| >| ASN | 211298 | @@ -282,9 +296,11 @@ This command retrieves Autonomous System Numbers (ASNs) associated with a domain | SilentPush.DomainASNs.asns | Unknown | Dictionary of Autonomous System Numbers \(ASNs\) associated with the domain. | ### **Command Example** + ```!silentpush-get-asns-for-domain domain="silentpush.com"``` ### **Context Example** + ```json { "domain": "silentpush.com", @@ -320,6 +336,7 @@ This command retrieves Autonomous System Numbers (ASNs) associated with a domain ### **Human Readable Output** >### Results + >| ASN | Description | >|---------|------------------------------------------| >| 13335 | CLOUDFLARENET, US | @@ -386,9 +403,11 @@ This command get certificate data collected from domain scanning. | SilentPush.Certificate.job_details.status | String | Status of the job. | ### **Command Example** + ```!silentpush-get-domain-certificates domain="silentpush.com"``` ### **Context Example** + ```json { "domain": "silentpush.com", @@ -409,6 +428,7 @@ This command get certificate data collected from domain scanning. ### **Human Readable Output** >### Result + >| Field | Value | >|------------------------------|----------------------------------------------| >| Common Name | silentpush.com | @@ -603,11 +623,13 @@ This command retrieves comprehensive enrichment information for a given resource | SilentPush.Enrichment.ip2asn.subnet_reputation_score | Number | A numerical risk score \(typically 0-100, with higher values indicating higher risk\). | ### **Command Example** + ```bash !silentpush-get-enrichment-data resource="ipv4" value="142.251.188.102" ``` ### **Context Example** + ```json { "resource": "ipv4", @@ -648,6 +670,7 @@ This command retrieves comprehensive enrichment information for a given resource ### **Human Readable Output** >### Result + >| Field | Value | >|--------------------------------|--------------------------| >| ASN | 15169 | @@ -782,11 +805,13 @@ This command fetch indicators of potential future attacks using a feed UUID. | SilentPush.FutureAttackIndicators.indicators.source_geographic_spread_explain | Unknown | Explanation of the geographic spread of the indicator as provided by the source. | ### **Command Example** + ```bash !silentpush-get-future-attack-indicators feed_uuid="99da9b6a-146b-4a4d-9929-5fd5c6e2c257" ``` ### **Context Example** + ```json { "feed_uuid": "99da9b6a-146b-4a4d-9929-5fd5c6e2c257", @@ -812,6 +837,7 @@ This command fetch indicators of potential future attacks using a feed UUID. ### **Human Readable Output** >### Result + >| Field | Value | >|-------------------------------|--------------------------------------------| >| Feed Name | capital-gainers.com | @@ -859,11 +885,13 @@ This command retrieve the reputation information for an IPv4. | SilentPush.IPv4Reputation.ip_reputation_explain.names_num_listed | Number | The number of domain names linked to this IP that are flagged or listed in security threat databases. | ### **Command Example** + ```bash !silentpush-get-nameserver-reputation nameserver="a.dns-servers.net.ru" limit="5" ``` ### **Context Example** + ```json { "nameserver": "a.dns-servers.net.ru", @@ -879,6 +907,7 @@ This command retrieve the reputation information for an IPv4. ### **Human Readable Output** >### Result + >| Field | Value | >|-------------------------|------------------------------| >| Nameserver | a.dns-servers.net.ru | @@ -915,11 +944,13 @@ This command retrieve status of running job or results from completed job. | SilentPush.JobStatus.status | String | Current status of the job. | ### **Command Example** + ```bash !silentpush-get-job-status job_id="d4067541-eafb-424c-98d3-de12d7a91331" ``` ### **Context Example** + ```json { "job_id": "d4067541-eafb-424c-98d3-de12d7a91331", @@ -933,6 +964,7 @@ This command retrieve status of running job or results from completed job. ### **Human Readable Output** >### Result + >| Field | Value | >|------------|-----------------------------------------| >| Job ID | d4067541-eafb-424c-98d3-de12d7a91331 | @@ -960,19 +992,21 @@ This command retrieve historical reputation data for a specified nameserver, inc | **Path** | **Type** | **Description** | | --- | --- | --- | -| SilentPush.SubnetReputation.nameserver | Number | The nameserver associated with the reputation history entry. | -| SilentPush.SubnetReputation.reputation_data.date | Number | Date of the reputation history entry \(in YYYYMMDD format\). | -| SilentPush.SubnetReputation.reputation_data.ns_server | String | Name of the nameserver associated with the reputation history entry. | -| SilentPush.SubnetReputation.reputation_data.ns_server_reputation | Number | Reputation score of the nameserver on the specified date. | -| SilentPush.SubnetReputation.reputation_data.ns_server_reputation_explain.ns_server_domain_density | Number | Number of domains associated with the nameserver. | -| SilentPush.SubnetReputation.reputation_data.ns_server_reputation_explain.ns_server_domains_listed | Number | Number of domains listed in reputation databases. | +| SilentPush.NameserverReputation.nameserver | Number | The nameserver associated with the reputation history entry. | +| SilentPush.NameserverReputation.reputation_data.date | Number | Date of the reputation history entry \(in YYYYMMDD format\). | +| SilentPush.NameserverReputation.reputation_data.ns_server | String | Name of the nameserver associated with the reputation history entry. | +| SilentPush.NameserverReputation.reputation_data.ns_server_reputation | Number | Reputation score of the nameserver on the specified date. | +| SilentPush.NameserverReputation.reputation_data.ns_server_reputation_explain.ns_server_domain_density | Number | Number of domains associated with the nameserver. | +| SilentPush.NameserverReputation.reputation_data.ns_server_reputation_explain.ns_server_domains_listed | Number | Number of domains listed in reputation databases. | ### **Command Example** + ```bash !silentpush-get-nameserver-reputation nameserver="a.dns-servers.net.ru" limit="5" ``` ### **Context Example** + ```json { "nameserver": "a.dns-servers.net.ru", @@ -988,6 +1022,7 @@ This command retrieve historical reputation data for a specified nameserver, inc ### **Human Readable Output** >### Result + >| Field | Value | >|-------------------------|------------------------------| >| Nameserver | a.dns-servers.net.ru | @@ -1026,11 +1061,13 @@ This command retrieves the reputation history for a specific subnet. | SilentPush.SubnetReputation.reputation_history.subnet_reputation_explain.ips_num_listed | Number | Number of listed IPs in the subnet. | ### **Command Example** + ```bash !silentpush-get-subnet-reputation subnet="192.168.0.0/16" ``` ### **Context Example** + ```json { "subnet": "192.168.0.0/16", @@ -1045,6 +1082,7 @@ This command retrieves the reputation history for a specific subnet. ### **Human Readable Output** >### Result + >| Field | Value | >|---------------------|--------------------------| >| Subnet | 192.168.0.0/16 | @@ -1088,11 +1126,13 @@ This command get domain information along with Silent Push risk score and live w | SilentPush.Domain.age | Number | The age of the domain in days. | ### **Command Example** + ```bash !silentpush-list-domain-information domains="silentpush.com" ``` ### **Context Example** + ```json { "domains": ["silentpush.com"], @@ -1115,6 +1155,7 @@ This command get domain information along with Silent Push risk score and live w ### **Human Readable Output** >### Results + >| Field | Value | >|-----------------------|------------------------------| >| Domain | silentpush.com | @@ -1166,11 +1207,13 @@ This command get infratags for multiple domains with optional clustering. | SilentPush.InfraTags.tag_clusters.100.match | String | The match string associated with the domains in the tag cluster with score 100. | ### **Command Example** + ```bash !silentpush-list-domain-infratags domains="silentpush.com" mode="live" match="self" as_of="self" ``` ### **Context Example** + ```json { "domains": ["silentpush.com"], @@ -1188,6 +1231,7 @@ This command get infratags for multiple domains with optional clustering. ### **Human Readable Output** >### Results + >| Field | Value | >|---------|------------------------------------------| >| Domain | silentpush.com | @@ -1274,11 +1318,13 @@ This command get IP information for multiple IPv4s and IPv6s. | SilentPush.IPInformation.ip_flags.vpn_tags | Unknown | List of VPN-related tags or null if not a VPN. | ### **Command Example** + ```bash !silentpush-list-ip-information ips="142.251.188.102" ``` ### **Context Example** + ```json { "ips": ["142.251.188.102"], @@ -1318,6 +1364,7 @@ This command get IP information for multiple IPv4s and IPv6s. ### **Human Readable Output** >### Results + >| Field | Value | >|-----------------------------------------|--------------------------------------------| >| ASN | 15169 | @@ -1459,11 +1506,13 @@ This command scan a URL to retrieve hosting metadata.. | SilentPush.URLScan.body_analysis.js_ssdeep | Unknown | List of ssdeep fuzzy hashes of JavaScript files. | ### **Command Example** + ```bash !silentpush-live-url-scan url="https://silentpush.com" ``` ### **Context Example** + ```json { "url": "https://silentpush.com", @@ -1477,9 +1526,10 @@ This command scan a URL to retrieve hosting metadata.. ### **Human Readable Output** >### Results + >| Field | Value | >|----------------|----------------------------| ->| URL | https://silentpush.com | +>| URL | | >| Scan Status | No scan results found | @@ -1530,11 +1580,13 @@ This command retrieve reverse Passive DNS data for specific DNS record types. | SilentPush.ReversePADNSLookup.records.type | String | The type of DNS record \(e.g., NS\). | ### **Command Example** + ```bash !silentpush-reverse-padns-lookup qtype="ns" qname="vida.ns.cloudflare.com" ``` ### **Context Example** + ```json { "qtype": "ns", @@ -1553,6 +1605,7 @@ This command retrieve reverse Passive DNS data for specific DNS record types. ### **Human Readable Output** >### Results + >| Field | Value | >|-------------------|--------------------------------------| >| Answer | vida.ns.cloudflare.com | @@ -1590,11 +1643,13 @@ This commandGenerate screenshot of a URL. | SilentPush.Screenshot.url | String | The URL that was used to generate the screenshot. | ### **Command Example** + ```bash !silentpush-screenshot-url url="https://www.virustotal.com/gui/domain/tbibank-bg.com" ``` ### **Context Example** + ```json { "url": "https://www.virustotal.com/gui/domain/tbibank-bg.com", @@ -1609,9 +1664,10 @@ This commandGenerate screenshot of a URL. ### **Human Readable Output** >### Results + >| Field | Value | >|-------------------|-------------------------------------------------------------------| ->| URL | https://www.virustotal.com/gui/domain/tbibank-bg.com | +>| URL | | >| Status | Success | >| Screenshot URL | [View Screenshot](https://fs.silentpush.com/screenshots/virustotal.com/f2fa9440ee769ad6f6702529c006522b.jpg) | >| File Name | www.virustotal.com_screenshot.jpg | @@ -1653,11 +1709,13 @@ This command search for domains with optional filters. | SilentPush.Domain.ip_diversity_groups | Number | The number of unique IP groups associated with the domain. | ### **Command Example** + ```bash !silentpush-search-domains ``` ### **Context Example** + ```json { "domain_search_results": [ @@ -1674,6 +1732,7 @@ This command search for domains with optional filters. ### **Human Readable Output** >### Results + >| Field | Value | >|----------------------|------------------------------------| >| ASN Diversity | 1 | @@ -1772,11 +1831,13 @@ This command search Silent Push scan data repositories using SPQL queries. | SilentPush.ScanData.url | String | The URL scanned. | ### **Command Example** + ```bash !silentpush-search-scan-data query="tld=cool" limit="5" ``` ### **Context Example** + ```json { "query": "tld=cool", @@ -1803,9 +1864,11 @@ This command search Silent Push scan data repositories using SPQL queries. ] } ``` + ### **Human Readable Output** >### Results + >| Field | Value | >|-------------------------|--------------------------------------------| >| Domain | [volunteering.cool](http://volunteering.cool) | diff --git a/Packs/SilentPush/Integrations/SilentPush/SilentPush.py b/Packs/SilentPush/Integrations/SilentPush/SilentPush.py index 783fb9045075..b8ccd7908c9e 100644 --- a/Packs/SilentPush/Integrations/SilentPush/SilentPush.py +++ b/Packs/SilentPush/Integrations/SilentPush/SilentPush.py @@ -6,13 +6,12 @@ import traceback from typing import Any import ast -from urllib.parse import urlencode +from urllib.parse import urlencode, urlparse import demistomock as demisto # noqa: E402 lgtm [py/polluting-import] from CommonServerPython import * # noqa: E402 lgtm [py/polluting-import] from CommonServerUserPython import * # noqa: E402 lgtm [py/polluting-import] - # Disable insecure warnings urllib3.disable_warnings() @@ -24,7 +23,7 @@ # API ENDPOINTS JOB_STATUS = "explore/job" -NAMESERVER_REPUTATION = "explore/nsreputation/nameserver" +NAMESERVER_REPUTATION = "explore/nsreputation/history/nameserver" SUBNET_REPUTATION = "explore/ipreputation/history/subnet" ASNS_DOMAIN = "explore/padns/lookup/domain/asns" DENSITY_LOOKUP = "explore/padns/lookup/density" @@ -57,10 +56,12 @@ InputArgument(name="max_wait", description="Number of seconds to wait for results (0-25 seconds)."), InputArgument(name="status_only", description="Return job status, even if job is complete."), InputArgument( - name="force_metadata_on", description="Always return query metadata, even if original request did not include metadata." + name="force_metadata_on", + description="Always return query metadata, even if original request did not include metadata.", ), InputArgument( - name="force_metadata_off", description="Never return query metadata, even if original request did include metadata." + name="force_metadata_off", + description="Never return query metadata, even if original request did include metadata.", ), ] NAMESERVER_REPUTATION_INPUTS = [ @@ -148,7 +149,9 @@ ] ENRICHMENT_INPUTS = [ InputArgument( - name="resource", description="Type of resource for which information needs to be retrieved {e.g. domain}.", required=True + name="resource", + description="Type of resource for which information needs to be retrieved {e.g. domain}.", + required=True, ), InputArgument( name="value", @@ -258,7 +261,9 @@ name="nameserver", output_type=int, description="The nameserver associated with the reputation history entry." ), OutputArgument( - name="reputation_data.date", output_type=int, description="Date of the reputation history entry (in YYYYMMDD format)." + name="reputation_data.date", + output_type=int, + description="Date of the reputation history entry (in YYYYMMDD format).", ), OutputArgument( name="reputation_data.ns_server", @@ -285,7 +290,9 @@ OutputArgument(name="subnet", output_type=str, description="The subnet associated with the reputation history."), OutputArgument(name="reputation_history.date", output_type=int, description="The date of the subnet reputation record."), OutputArgument( - name="reputation_history.subnet", output_type=str, description="The subnet associated with the reputation record." + name="reputation_history.subnet", + output_type=str, + description="The subnet associated with the reputation record.", ), OutputArgument( name="reputation_history.subnet_reputation", output_type=int, description="The reputation score of the subnet." @@ -309,13 +316,17 @@ ASNS_DOMAIN_OUTPUTS = [ OutputArgument(name="domain", output_type=str, description="The domain name for which ASNs are retrieved."), OutputArgument( - name="asns", output_type=dict, description="Dictionary of Autonomous System Numbers (ASNs) associated with the domain." + name="asns", + output_type=dict, + description="Dictionary of Autonomous System Numbers (ASNs) associated with the domain.", ), ] DENSITY_LOOKUP_OUTPUTS = [ OutputArgument(name="qtype", output_type=str, description="The following qtypes are supported: nssrv, mxsrv."), OutputArgument( - name="query", output_type=str, description="The query value to lookup, which can be the name of an NS or MX server." + name="query", + output_type=str, + description="The query value to lookup, which can be the name of an NS or MX server.", ), OutputArgument(name="records.density", output_type=int, description="The density value associated with the query result."), OutputArgument(name="records.nssrv", output_type=str, description="The name server (NS) for the query result."), @@ -328,10 +339,14 @@ ), OutputArgument(name="host", output_type=str, description="The domain name (host) associated with the record."), OutputArgument( - name="ip_diversity_all", output_type=int, description="The total number of unique IPs associated with the domain." + name="ip_diversity_all", + output_type=int, + description="The total number of unique IPs associated with the domain.", ), OutputArgument( - name="ip_diversity_groups", output_type=int, description="The number of unique IP groups associated with the domain." + name="ip_diversity_groups", + output_type=int, + description="The number of unique IP groups associated with the domain.", ), ] DOMAIN_INFRATAGS_OUTPUTS = [ @@ -339,7 +354,9 @@ OutputArgument(name="infratags.mode", output_type=str, description="The mode associated with the domain infratag."), OutputArgument(name="infratags.tag", output_type=str, description="The tag associated with the domain infratag."), OutputArgument( - name="tag_clusters.25.domains", output_type=list, description="List of domains in the tag cluster with score 25." + name="tag_clusters.25.domains", + output_type=list, + description="List of domains in the tag cluster with score 25.", ), OutputArgument( name="tag_clusters.25.match", @@ -347,7 +364,9 @@ description="The match string associated with the domains in the tag cluster with score 25.", ), OutputArgument( - name="tag_clusters.50.domains", output_type=list, description="List of domains in the tag cluster with score 50." + name="tag_clusters.50.domains", + output_type=list, + description="List of domains in the tag cluster with score 50.", ), OutputArgument( name="tag_clusters.50.match", @@ -355,7 +374,9 @@ description="The match string associated with the domains in the tag cluster with score 50.", ), OutputArgument( - name="tag_clusters.75.domains", output_type=list, description="List of domains in the tag cluster with score 75." + name="tag_clusters.75.domains", + output_type=list, + description="List of domains in the tag cluster with score 75.", ), OutputArgument( name="tag_clusters.75.match", @@ -363,7 +384,9 @@ description="The match string associated with the domains in the tag cluster with score 75.", ), OutputArgument( - name="tag_clusters.100.domains", output_type=list, description="List of domains in the tag cluster with score 100." + name="tag_clusters.100.domains", + output_type=list, + description="List of domains in the tag cluster with score 100.", ), OutputArgument( name="tag_clusters.100.match", @@ -416,7 +439,9 @@ OutputArgument(name="certificates.source_url", output_type=str, description="URL of the certificate log source."), OutputArgument(name="certificates.subject", output_type=str, description="Subject details of the certificate."), OutputArgument( - name="certificates.wildcard", output_type=int, description="Indicates if the certificate is a wildcard certificate." + name="certificates.wildcard", + output_type=int, + description="Indicates if the certificate is a wildcard certificate.", ), OutputArgument(name="job_details.get", output_type=str, description="URL to get the data of the job or its status."), OutputArgument(name="job_details.job_id", output_type=str, description="ID of the job."), @@ -458,7 +483,9 @@ description="Score indicating likelihood of domain being dynamically generated.", ), OutputArgument( - name="domain_urls.results_summary.is_dynamic_domain", output_type=bool, description="Indicates if the domain is dynamic." + name="domain_urls.results_summary.is_dynamic_domain", + output_type=bool, + description="Indicates if the domain is dynamic.", ), OutputArgument( name="domain_urls.results_summary.is_url_shortener", @@ -466,7 +493,9 @@ description="Indicates if the domain is a known URL shortener.", ), OutputArgument( - name="domain_urls.results_summary.results", output_type=int, description="Number of results found for the domain." + name="domain_urls.results_summary.results", + output_type=int, + description="Number of results found for the domain.", ), OutputArgument( name="domain_urls.results_summary.url_shortner_score", output_type=int, description="Score of the shortned URL" @@ -479,14 +508,20 @@ OutputArgument(name="domaininfo.whois_created_date", output_type=str, description="The created date on WHOIS records."), OutputArgument(name="domaininfo.query", output_type=str, description="The domain name that was queried in the system."), OutputArgument( - name="domaininfo.last_seen", output_type=int, description="The first recorded observation of the domain in the database." + name="domaininfo.last_seen", + output_type=int, + description="The first recorded observation of the domain in the database.", ), OutputArgument( - name="domaininfo.first_seen", output_type=int, description="The last recorded observation of the domain in the database." + name="domaininfo.first_seen", + output_type=int, + description="The last recorded observation of the domain in the database.", ), OutputArgument(name="domaininfo.is_new", output_type=bool, description='Indicates whether the domain is considered "new.".'), OutputArgument( - name="domaininfo.is_new_score", output_type=int, description='A scoring metric indicating how "new" the domain is.' + name="domaininfo.is_new_score", + output_type=int, + description='A scoring metric indicating how "new" the domain is.', ), OutputArgument(name="domaininfo.age", output_type=int, description="Represents the age of the domain in days."), OutputArgument( @@ -495,10 +530,14 @@ description="A scoring metric indicating the trustworthiness of the domain based on its age.", ), OutputArgument( - name="ip_diversity.asn_diversity", output_type=str, description="Number of different ASNs associated with the domain." + name="ip_diversity.asn_diversity", + output_type=str, + description="Number of different ASNs associated with the domain.", ), OutputArgument( - name="ip_diversity.ip_diversity_all", output_type=str, description="Total number of unique IPs observed for the domain." + name="ip_diversity.ip_diversity_all", + output_type=str, + description="Total number of unique IPs observed for the domain.", ), OutputArgument(name="ip_diversity.host", output_type=str, description="The hostname being analyzed."), OutputArgument( @@ -507,7 +546,9 @@ description="The number of distinct IP groups (e.g., IPs belonging to different ranges or providers).", ), OutputArgument( - name="ns_reputation.is_expired", output_type=bool, description="Indicates if the domain`s nameserver is expired." + name="ns_reputation.is_expired", + output_type=bool, + description="Indicates if the domain`s nameserver is expired.", ), OutputArgument( name="ns_reputation.is_parked", @@ -523,7 +564,9 @@ name="ns_reputation.ns_reputation_max", output_type=int, description="Maximum reputation score for nameservers." ), OutputArgument( - name="ns_reputation.ns_reputation_score", output_type=int, description="Reputation score of the domain`s nameservers." + name="ns_reputation.ns_reputation_score", + output_type=int, + description="Reputation score of the domain`s nameservers.", ), OutputArgument(name="ns_reputation.ns_srv_reputation.domain", output_type=str, description="The nameservers of domain."), OutputArgument(name="ns_reputation.ns_srv_reputation.ns_server", output_type=str, description="Provided nameserver."), @@ -538,10 +581,14 @@ description="Number of listed domains using this NS.", ), OutputArgument( - name="ns_reputation.ns_srv_reputation.ns_server_reputation", output_type=int, description="Reputation score for this NS" + name="ns_reputation.ns_srv_reputation.ns_server_reputation", + output_type=int, + description="Reputation score for this NS", ), OutputArgument( - name="scan_data.certificates.domain", output_type=str, description="Domain for which the SSL certificate was issued." + name="scan_data.certificates.domain", + output_type=str, + description="Domain for which the SSL certificate was issued.", ), OutputArgument( name="scan_data.certificates.domains", @@ -554,16 +601,24 @@ description="Issuer organization of the SSL certificate.", ), OutputArgument( - name="scan_data.certificates.fingerprint_sha1", output_type=str, description="A unique identifier for the certificate." + name="scan_data.certificates.fingerprint_sha1", + output_type=str, + description="A unique identifier for the certificate.", ), OutputArgument( - name="scan_data.certificates.hostname", output_type=str, description="The hostname associated with the certificate." + name="scan_data.certificates.hostname", + output_type=str, + description="The hostname associated with the certificate.", ), OutputArgument( - name="scan_data.certificates.ip", output_type=str, description="The IP address of the server using this certificate." + name="scan_data.certificates.ip", + output_type=str, + description="The IP address of the server using this certificate.", ), OutputArgument( - name="scan_data.certificates.is_expired", output_type=str, description="Indicates whether the certificate has expired." + name="scan_data.certificates.is_expired", + output_type=str, + description="Indicates whether the certificate has expired.", ), OutputArgument( name="scan_data.certificates.issuer_common_name", @@ -585,7 +640,9 @@ OutputArgument(name="scan_data.headers.scan_date", output_type=str, description="The date when the headers were scanned."), OutputArgument(name="scan_data.headers.headers.cache-control", output_type=str, description="HTTP cache-control"), OutputArgument( - name='scan_data.headers.headers.content-length"', output_type=str, description="Content lenght of the HTTP response." + name='scan_data.headers.headers.content-length"', + output_type=str, + description="Content lenght of the HTTP response.", ), OutputArgument(name="scan_data.headers.headers.date", output_type=str, description="The date/time of the response."), OutputArgument( @@ -623,7 +680,9 @@ OutputArgument(name="scan_data.jarm.hostname", output_type=str, description="The hostname where this jarm was found."), OutputArgument(name="scan_data.jarm.ip", output_type=str, description="The IP address responding to the request."), OutputArgument( - name="scan_data.jarm.jarm_hash", output_type=str, description="Unique identifier for the TLS configuration of the server." + name="scan_data.jarm.jarm_hash", + output_type=str, + description="Unique identifier for the TLS configuration of the server.", ), OutputArgument(name="scan_data.jarm.scan_date", output_type=str, description="Date when this jarm was last scanned."), OutputArgument(name="sp_risk_score", output_type=int, description="Overall risk score for the domain."), @@ -642,10 +701,14 @@ name="ip2asn.asn_reputation_explain.ips_in_asn", output_type=int, description="Total number of IPs in the ASN." ), OutputArgument( - name="ip2asn.asn_reputation_explain.ips_num_active", output_type=int, description="Number of active IPs in the ASN." + name="ip2asn.asn_reputation_explain.ips_num_active", + output_type=int, + description="Number of active IPs in the ASN.", ), OutputArgument( - name="ip2asn.asn_reputation_explain.ips_num_listed", output_type=int, description="Number of listed IPs in the ASN." + name="ip2asn.asn_reputation_explain.ips_num_listed", + output_type=int, + description="Number of listed IPs in the ASN.", ), OutputArgument(name="ip2asn.asn_reputation_score", output_type=int, description="Reputation score of the ASN."), OutputArgument(name="ip2asn.asn_takedown_reputation", output_type=int, description="Takedown reputation score of the ASN."), @@ -670,7 +733,9 @@ description="Maximum age of listings for the ASN with takedown reputation.", ), OutputArgument( - name="ip2asn.asn_takedown_reputation_score", output_type=int, description="Takedown reputation score of the ASN." + name="ip2asn.asn_takedown_reputation_score", + output_type=int, + description="Takedown reputation score of the ASN.", ), OutputArgument(name="ip2asn.asname", output_type=str, description="Name of the Autonomous System (AS)."), OutputArgument( @@ -704,7 +769,9 @@ ), OutputArgument(name="ip2asn.ip_is_dsl_dynamic", output_type=bool, description="the IP is from a dynamic DSL pool."), OutputArgument( - name="ip2asn.ip_is_dsl_dynamic_score", output_type=int, description="A score indicating how likely this IP is dynamic." + name="ip2asn.ip_is_dsl_dynamic_score", + output_type=int, + description="A score indicating how likely this IP is dynamic.", ), OutputArgument( name="ip2asn.ip_is_ipfs_node", @@ -712,7 +779,9 @@ description="the InterPlanetary File System (IPFS), a decentralized file storage system.", ), OutputArgument( - name="ip2asn.ip_is_tor_exit_node", output_type=bool, description="Tor exit node (used for anonymous internet browsing)." + name="ip2asn.ip_is_tor_exit_node", + output_type=bool, + description="Tor exit node (used for anonymous internet browsing).", ), OutputArgument( name="ip2asn.ip_location.continent_code", @@ -742,7 +811,9 @@ description="Measures how frequently the IP appears in threat intelligence or blacklist databases.", ), OutputArgument( - name="ip2asn.listing_score_explain", output_type=dict, description="A breakdown of why the listing score is assigned." + name="ip2asn.listing_score_explain", + output_type=dict, + description="A breakdown of why the listing score is assigned.", ), OutputArgument(name="ip2asn.malscore", output_type=int, description="Malicious activity score for the IP."), OutputArgument( @@ -771,7 +842,9 @@ description="Organization that issued the SSL certificate.", ), OutputArgument( - name="ip2asn.scan_data.certificates.not_before", output_type=str, description="Start date of SSL certificate validity." + name="ip2asn.scan_data.certificates.not_before", + output_type=str, + description="Start date of SSL certificate validity.", ), OutputArgument( name="ip2asn.scan_data.certificates.not_after", @@ -787,7 +860,9 @@ OutputArgument(name="ip2asn.scan_data.certificates.scan_date", output_type=str, description="Scan date of the certificate."), OutputArgument(name="ip2asn.scan_data.favicon.favicon2_md5", output_type=str, description="MD5 hash of the second favicon."), OutputArgument( - name="ip2asn.scan_data.favicon.favicon2_mmh3", output_type=int, description="MurmurHash3 value of the second favicon." + name="ip2asn.scan_data.favicon.favicon2_mmh3", + output_type=int, + description="MurmurHash3 value of the second favicon.", ), OutputArgument(name="ip2asn.scan_data.favicon.favicon_md5", output_type=str, description="MD5 hash of the favicon."), OutputArgument( @@ -799,10 +874,14 @@ OutputArgument(name="ip2asn.scan_data.favicon.scan_date", output_type=str, description="Scan date of favicon file."), OutputArgument(name="ip2asn.scan_data.headers.response", output_type=str, description="HTTP response code from the scan."), OutputArgument( - name="ip2asn.scan_data.headers.scan_date", output_type=str, description="The date and time when the scan was performed." + name="ip2asn.scan_data.headers.scan_date", + output_type=str, + description="The date and time when the scan was performed.", ), OutputArgument( - name="ip2asn.scan_data.headers.headers.server", output_type=str, description="Server header from the HTTP response." + name="ip2asn.scan_data.headers.headers.server", + output_type=str, + description="Server header from the HTTP response.", ), OutputArgument( name="ip2asn.scan_data.headers.headers.content-type", @@ -824,16 +903,24 @@ ), OutputArgument(name="ip2asn.scan_data.html.html_title", output_type=str, description="Title of the scanned HTML page."), OutputArgument( - name="ip2asn.scan_data.html.html_body_murmur3", output_type=str, description="MurmurHash3 of the HTML body content." + name="ip2asn.scan_data.html.html_body_murmur3", + output_type=str, + description="MurmurHash3 of the HTML body content.", ), OutputArgument( - name="ip2asn.scan_data.html.html_body_ssdeep", output_type=str, description="SSDEEP fuzzy hash of the HTML body content." + name="ip2asn.scan_data.html.html_body_ssdeep", + output_type=str, + description="SSDEEP fuzzy hash of the HTML body content.", ), OutputArgument( - name="ip2asn.scan_data.html.scan_date", output_type=str, description="The date and time when the scan was performed." + name="ip2asn.scan_data.html.scan_date", + output_type=str, + description="The date and time when the scan was performed.", ), OutputArgument( - name="ip2asn.scan_data.jarm.scan_date", output_type=str, description="The date and time when the scan was performed." + name="ip2asn.scan_data.jarm.scan_date", + output_type=str, + description="The date and time when the scan was performed.", ), OutputArgument( name="ip2asn.scan_data.jarm.jarm_hash", output_type=str, description="JARM fingerprint hash for TLS analysis." @@ -884,7 +971,9 @@ LIST_IP_OUTPUTS = [ OutputArgument(name="ip_is_dsl_dynamic", output_type=bool, description="Indicates if the IP is a DSL dynamic IP."), OutputArgument( - name="ip_has_expired_certificate", output_type=bool, description="Indicates if the IP has an expired certificate." + name="ip_has_expired_certificate", + output_type=bool, + description="Indicates if the IP has an expired certificate.", ), OutputArgument(name="subnet_allocation_age", output_type=str, description="Age of the subnet allocation."), OutputArgument(name="asn_rank_score", output_type=int, description="Rank score of the ASN."), @@ -896,7 +985,9 @@ description="Number of active IPs in the ASN takedown reputation.", ), OutputArgument( - name="asn_takedown_reputation_explain.ips_in_asn", output_type=int, description="Total number of IPs in the ASN." + name="asn_takedown_reputation_explain.ips_in_asn", + output_type=int, + description="Total number of IPs in the ASN.", ), OutputArgument( name="asn_takedown_reputation_explain.ips_num_listed", @@ -960,7 +1051,9 @@ OutputArgument(name="listing_score", output_type=int, description="Listing score of the IP."), OutputArgument(name="malscore", output_type=int, description="Malware score associated with the IP."), OutputArgument( - name="sinkhole_info.known_sinkhole_ip", output_type=bool, description="Indicates if the IP is a known sinkhole IP." + name="sinkhole_info.known_sinkhole_ip", + output_type=bool, + description="Indicates if the IP is a known sinkhole IP.", ), OutputArgument(name="sinkhole_info.tags", output_type=str, description="Tags associated with the sinkhole information."), OutputArgument(name="subnet_reputation", output_type=int, description="Reputation score of the subnet."), @@ -985,12 +1078,16 @@ ] ASN_REPUTATION_OUTPUTS = [ OutputArgument( - name="asn", output_type=int, description="Autonomous System Number (ASN) associated with the reputation history." + name="asn", + output_type=int, + description="Autonomous System Number (ASN) associated with the reputation history.", ), OutputArgument(name="asn_reputation", output_type=int, description="Reputation score of the ASN at a given point in time."), OutputArgument(name="asn_reputation_explain.ips_in_asn", output_type=int, description="Total number of IPs within the ASN."), OutputArgument( - name="asn_reputation_explain.ips_num_active", output_type=int, description="Number of actively used IPs in the ASN." + name="asn_reputation_explain.ips_num_active", + output_type=int, + description="Number of actively used IPs in the ASN.", ), OutputArgument( name="asn_reputation_explain.ips_num_listed", @@ -1007,7 +1104,9 @@ name="takedown_reputation.allocation_age", output_type=int, description="The age of the ASN allocation in days." ), OutputArgument( - name="takedown_reputation.allocation_date", output_type=int, description="The date when the ASN was allocated (YYYYMMDD)." + name="takedown_reputation.allocation_date", + output_type=int, + description="The date when the ASN was allocated (YYYYMMDD).", ), OutputArgument( name="takedown_reputation.asn_takedown_reputation", @@ -1097,10 +1196,14 @@ OutputArgument(name="body_analysis.google-adstag", output_type=list, description="List of Google adstag identifiers."), OutputArgument(name="body_analysis.header_sha256", output_type=list, description="SHA-256 hash of the header content."), OutputArgument( - name="body_analysis.js_sha256", output_type=list, description="List of JavaScript files with SHA-256 hash values." + name="body_analysis.js_sha256", + output_type=list, + description="List of JavaScript files with SHA-256 hash values.", ), OutputArgument( - name="body_analysis.js_ssdeep", output_type=list, description="List of JavaScript files with SSDEEP hash values." + name="body_analysis.js_ssdeep", + output_type=list, + description="List of JavaScript files with SSDEEP hash values.", ), OutputArgument(name="body_analysis.onion", output_type=list, description="List of Onion URLs detected."), OutputArgument(name="body_analysis.telegram", output_type=list, description="List of Telegram-related information."), @@ -1206,13 +1309,17 @@ OutputArgument(name="origin_ssl.not_after", output_type=str, description="Expiration date of the SSL certificate."), OutputArgument(name="origin_ssl.not_before", output_type=str, description="Start date of the SSL certificate validity."), OutputArgument( - name="origin_ssl.sans", output_type=list, description="List of Subject Alternative Names (SANs) for the SSL certificate." + name="origin_ssl.sans", + output_type=list, + description="List of Subject Alternative Names (SANs) for the SSL certificate.", ), OutputArgument(name="origin_ssl.sans_count", output_type=int, description="Count of SANs for the SSL certificate."), OutputArgument(name="origin_ssl.serial_number", output_type=str, description="Serial number of the SSL certificate."), OutputArgument(name="origin_ssl.sigalg", output_type=str, description="Signature algorithm used for the SSL certificate."), OutputArgument( - name="origin_ssl.subject.common_name", output_type=str, description="Subject common name for the SSL certificate." + name="origin_ssl.subject.common_name", + output_type=str, + description="Subject common name for the SSL certificate.", ), OutputArgument(name="origin_ssl.subject_key_id", output_type=str, description="Subject Key Identifier for SSL certificate."), OutputArgument(name="origin_ssl.valid", output_type=bool, description="Indicates if the SSL certificate is valid."), @@ -1241,7 +1348,9 @@ OutputArgument(name="ssl.not_after", output_type=str, description="Expiration date of the SSL certificate."), OutputArgument(name="ssl.not_before", output_type=str, description="Start date of the SSL certificate validity."), OutputArgument( - name="ssl.sans", output_type=list, description="List of Subject Alternative Names (SANs) for the SSL certificate." + name="ssl.sans", + output_type=list, + description="List of Subject Alternative Names (SANs) for the SSL certificate.", ), OutputArgument(name="ssl.sans_count", output_type=int, description="Count of SANs for the SSL certificate."), OutputArgument(name="ssl.serial_number", output_type=str, description="Serial number of the SSL certificate."), @@ -1280,17 +1389,25 @@ description="Cumulative score assigned to the indicator by all sources.", ), OutputArgument( - name="indicators.name", output_type=str, description="Name associated with the indicator, such as a domain name." + name="indicators.name", + output_type=str, + description="Name associated with the indicator, such as a domain name.", ), OutputArgument( - name="indicators.total_custom", output_type=int, description="Total number of custom indicators for the specific entry." + name="indicators.total_custom", + output_type=int, + description="Total number of custom indicators for the specific entry.", ), OutputArgument(name="indicators.source_name", output_type=str, description="Name of the source providing the indicator."), OutputArgument( - name="indicators.first_seen_on", output_type=str, description="Date and time when the indicator was first observed." + name="indicators.first_seen_on", + output_type=str, + description="Date and time when the indicator was first observed.", ), OutputArgument( - name="indicators.last_seen_on", output_type=str, description="Date and time when the indicator was last observed." + name="indicators.last_seen_on", + output_type=str, + description="Date and time when the indicator was last observed.", ), OutputArgument(name="indicators.type", output_type=str, description="Type of the indicator (e.g., domain, IP address, URL)."), OutputArgument(name="indicators.uuid", output_type=str, description="Unique identifier assigned to the indicator."), @@ -1300,7 +1417,9 @@ description="Template type describing the indicator (e.g., domain template).", ), OutputArgument( - name="indicators.ioc_uuid", output_type=str, description="Unique identifier for the IOC related to the indicator." + name="indicators.ioc_uuid", + output_type=str, + description="Unique identifier for the IOC related to the indicator.", ), OutputArgument( name="indicators.source_vendor_name", @@ -1332,7 +1451,9 @@ description="Indicates whether the IP address is a known TOR exit node.", ), OutputArgument( - name="indicators.ip_is_dsl_dynamic", output_type=bool, description="Indicates whether the IP address is a DSL dynamic IP." + name="indicators.ip_is_dsl_dynamic", + output_type=bool, + description="Indicates whether the IP address is a DSL dynamic IP.", ), OutputArgument( name="indicators.ip_reputation_score", @@ -1350,7 +1471,9 @@ description="Indicates whether the indicator is known to be benign or harmless.", ), OutputArgument( - name="indicators.asn_rank_score", output_type=int, description="Score indicating the reputation rank of the ASN." + name="indicators.asn_rank_score", + output_type=int, + description="Score indicating the reputation rank of the ASN.", ), OutputArgument( name="indicators.asn_reputation_score", @@ -1373,7 +1496,9 @@ description="Reputation score of the ASN considering takedown activities or abuse reports.", ), OutputArgument( - name="indicators.asn", output_type=int, description="Autonomous System Number (ASN) associated with the indicator." + name="indicators.asn", + output_type=int, + description="Autonomous System Number (ASN) associated with the indicator.", ), OutputArgument( name="indicators.density", @@ -1381,7 +1506,9 @@ description="Indicator density score based on traffic or other relevant factors.", ), OutputArgument( - name="indicators.asn_rank", output_type=int, description="Rank of the ASN indicating its reputation or trustworthiness." + name="indicators.asn_rank", + output_type=int, + description="Rank of the ASN indicating its reputation or trustworthiness.", ), OutputArgument( name="indicators.malscore", @@ -1401,14 +1528,20 @@ ), OutputArgument(name="indicators.ipv4", output_type=str, description="IPv4 address associated with the indicator."), OutputArgument( - name="indicators.asname", output_type=str, description="Autonomous System Name (ASName) associated with the ASN." + name="indicators.asname", + output_type=str, + description="Autonomous System Name (ASName) associated with the ASN.", ), OutputArgument( - name="indicators.ip_ptr", output_type=str, description="PTR (reverse DNS) record associated with the IP address." + name="indicators.ip_ptr", + output_type=str, + description="PTR (reverse DNS) record associated with the IP address.", ), OutputArgument(name="indicators.subnet", output_type=str, description="Subnet associated with the indicator."), OutputArgument( - name="indicators.country_code", output_type=int, description="Country code associated with the indicator (e.g., US, CA)." + name="indicators.country_code", + output_type=int, + description="Country code associated with the indicator (e.g., US, CA).", ), OutputArgument( name="indicators.continent_code", @@ -1416,7 +1549,9 @@ description="Continent code associated with the indicator (e.g., NA, EU).", ), OutputArgument( - name="indicators.it_exists", output_type=bool, description="Indicates if the indicator currently exists in the dataset." + name="indicators.it_exists", + output_type=bool, + description="Indicates if the indicator currently exists in the dataset.", ), OutputArgument(name="indicators.is_new", output_type=bool, description="Indicates if the indicator is newly detected."), OutputArgument( @@ -1425,7 +1560,9 @@ description="Indicates if the domain is part of the Alexa Top 10K list.", ), OutputArgument( - name="indicators.is_dynamic_domain", output_type=bool, description="Indicates if the domain is classified as dynamic." + name="indicators.is_dynamic_domain", + output_type=bool, + description="Indicates if the domain is classified as dynamic.", ), OutputArgument( name="indicators.is_url_shortener", @@ -1457,13 +1594,17 @@ description="Score indicating the likelihood of the domain being newly registered.", ), OutputArgument( - name="indicators.ns_avg_ttl_score", output_type=int, description="Score representing the average TTL of the nameservers." + name="indicators.ns_avg_ttl_score", + output_type=int, + description="Score representing the average TTL of the nameservers.", ), OutputArgument( name="indicators.ns_reputation_max", output_type=int, description="Maximum reputation score of the nameservers." ), OutputArgument( - name="indicators.ns_reputation_score", output_type=int, description="Overall reputation score of the nameservers." + name="indicators.ns_reputation_score", + output_type=int, + description="Overall reputation score of the nameservers.", ), OutputArgument( name="indicators.avg_probability_score", @@ -1495,7 +1636,9 @@ name="indicators.whois_age", output_type=int, description="Age of the domain based on the WHOIS creation date." ), OutputArgument( - name="indicators.alexa_rank", output_type=int, description="Alexa rank of the domain, indicating its popularity." + name="indicators.alexa_rank", + output_type=int, + description="Alexa rank of the domain, indicating its popularity.", ), OutputArgument( name="indicators.asn_diversity", @@ -1518,11 +1661,15 @@ description="Average probability indicating the likelihood of malicious activity.", ), OutputArgument( - name="indicators.whois_created_date", output_type=str, description="Creation date of the domain from WHOIS records." + name="indicators.whois_created_date", + output_type=str, + description="Creation date of the domain from WHOIS records.", ), OutputArgument(name="indicators.domain", output_type=str, description="Domain name associated with the indicator."), OutputArgument( - name="indicators.subdomain", output_type=str, description="Subdomain associated with the indicator, if applicable." + name="indicators.subdomain", + output_type=str, + description="Subdomain associated with the indicator, if applicable.", ), OutputArgument(name="indicators.host", output_type=str, description="Host associated with the indicator."), OutputArgument( @@ -1566,7 +1713,9 @@ description="Custom score provided by the source for the indicator.", ), OutputArgument( - name="indicators.source_score", output_type=int, description="Overall score assigned by the source to the indicator." + name="indicators.source_score", + output_type=int, + description="Overall score assigned by the source to the indicator.", ), OutputArgument( name="indicators.source_frequency", @@ -1617,7 +1766,12 @@ required=False, key_type=ParameterTypes.AUTH, ), - ConfKey(name="insecure", display="Trust any certificate (not secure)", required=False, key_type=ParameterTypes.BOOLEAN), + ConfKey( + name="insecure", + display="Trust any certificate (not secure)", + required=False, + key_type=ParameterTypes.BOOLEAN, + ), ConfKey(name="proxy", display="Use system proxy settings", required=False, key_type=ParameterTypes.BOOLEAN), ], ) @@ -1722,18 +1876,23 @@ def get_nameserver_reputation(self, nameserver: str, explain: bool = False, limi limit (int): Maximum number of reputation entries to return. Returns: - dict: Reputation history for the given nameserver. + list: A list of reputation entries (each being a dict) for the given nameserver. """ - url_suffix = f"{NAMESERVER_REPUTATION}/{nameserver}" - params = {"explain": explain, "limit": limit} + params = {"explain": int(bool(explain)), "limit": limit} + remove_nulls_from_dictionary(params) response = self._http_request(method="GET", url_suffix=url_suffix, params=params) - # Return the reputation history, or an empty list if not found - return response.get("response", {}).get("ns_server_reputation", []) + if isinstance(response, str): + try: + response = json.loads(response) + except Exception as e: + raise ValueError(f"Unable to parse JSON from response: {e}") + + return response.get("response", {}).get("ns_server_reputation_history", []) def get_subnet_reputation(self, subnet: str, explain: bool = False, limit: int | None = None) -> dict[str, Any]: """ @@ -1882,11 +2041,17 @@ def list_domain_infratags( """ url_suffix = DOMAIN_INFRATAGS - params = {"mode": mode, "match": match, "clusters": int(cluster), "as_of": as_of, "origin_uid": origin_uid} - remove_nulls_from_dictionary(params) + payload = { + "domains": domains, + "mode": mode, + "match": match, + "clusters": int(cluster), + "as_of": as_of, + "origin_uid": origin_uid, + } + remove_nulls_from_dictionary(payload) - payload = {"domains": domains} - return self._http_request(method="POST", url_suffix=url_suffix, params=params, data=payload) + return self._http_request(method="POST", url_suffix=url_suffix, data=payload) def fetch_bulk_domain_info(self, domains: list[str]) -> dict[str, Any]: """Fetch basic domain information for a list of domains.""" @@ -1909,9 +2074,11 @@ def fetch_whois_info(self, domain: str) -> dict[str, Any]: return { "Registrant Name": whois_data.get("name", "N/A"), "Registrant Organization": whois_data.get("org", "N/A"), - "Registrant Address": ", ".join(whois_data.get("address", [])) - if isinstance(whois_data.get("address"), list) - else whois_data.get("address", "N/A"), + "Registrant Address": ( + ", ".join(whois_data.get("address", [])) + if isinstance(whois_data.get("address"), list) + else whois_data.get("address", "N/A") + ), "Registrant City": whois_data.get("city", "N/A"), "Registrant State": whois_data.get("state", "N/A"), "Registrant Country": whois_data.get("country", "N/A"), @@ -2079,7 +2246,7 @@ def list_ip_information(self, ips: list[str], resource: str) -> dict: return self._http_request("POST", url_suffix, data=ip_data) - def get_asn_reputation(self, asn: int, limit: int | None = None, explain: bool | None = False) -> dict[str, Any]: + def get_asn_reputation(self, asn: int, limit: int | None = None, explain: bool = False) -> dict[str, Any]: """ Retrieve reputation history for a specific Autonomous System Number (ASN). @@ -2091,10 +2258,9 @@ def get_asn_reputation(self, asn: int, limit: int | None = None, explain: bool | Returns: Dict[str, Any]: ASN reputation history information. """ - url_suffix = f"{ASN_REPUTATION}/{asn}" - query_params = assign_params(limit=limit, explain=explain) + params = {"explain": int(bool(explain)), "limit": limit} - return self._http_request(method="GET", url_suffix=url_suffix, params=query_params) + return self._http_request(method="GET", url_suffix=f"{ASN_REPUTATION}/{asn}", params=params) def get_asn_takedown_reputation(self, asn: str, explain: int = 0, limit: int = None) -> dict[str, Any]: """ @@ -2115,29 +2281,50 @@ def get_asn_takedown_reputation(self, asn: str, explain: int = 0, limit: int = N """ if not asn: raise ValueError("ASN is required.") + url_suffix = f"{ASN_TAKEDOWN_REPUTATION}/{asn}" - query_params = assign_params(limit=limit, explain=explain) + query_params = assign_params(explain=int(bool(explain)), limit=limit) raw_response = self._http_request(method="GET", url_suffix=url_suffix, params=query_params) + return raw_response.get("response", {}) - response = raw_response.get("response") + def get_ipv4_reputation(self, ipv4: str, explain: bool = False, limit: int = None) -> dict: + """ + Retrieve historical reputation data for the specified IPv4 address. - if isinstance(response, dict): - return response.get("takedown_reputation", {}) - else: - # Log or wrap the unexpected response type for debugging or display - return {"error": response if isinstance(response, str) else "Unexpected response type"} + Args: + ipv4 (str): The IPv4 address to check. + explain (bool): Whether to include explanation details. + limit (int): Maximum number of history entries to return. - def get_ipv4_reputation(self, ipv4: str, explain: int = 0, limit: int = None) -> dict[str, Any]: - """ - Retrieve reputation information for an IPv4 address. + Returns: + dict: Dictionary containing 'ip_reputation_history' key with list of entries. """ url_suffix = f"{IPV4_REPUTATION}/{ipv4}" - query_params = assign_params(limit=limit, explain=explain) - raw_response = self._http_request(method="GET", url_suffix=url_suffix, params=query_params) + params = {"explain": int(bool(explain)), "limit": limit} + + remove_nulls_from_dictionary(params) + + response = self._http_request( + method="GET", + url_suffix=url_suffix, + params=params, + headers={"Accept": "application/json", "Content-Type": "application/json"}, + ) + + if response.get("error") is not None: + raise ValueError(f"API Error: {response['error']}") + + if isinstance(response, str): + try: + response = json.loads(response) + except Exception as e: + raise ValueError(f"Unable to parse JSON from response: {e}") - return raw_response + data = response.get("response", {}).get("ip_reputation_history", []) + + return {"ip_reputation_history": data if isinstance(data, list) else []} def forward_padns_lookup(self, qtype: str, qname: str, **kwargs) -> dict[str, Any]: """ @@ -2205,7 +2392,12 @@ def search_scan_data(self, query: str, params: dict) -> dict[str, Any]: return self._http_request(method="POST", url_suffix=url_suffix, data=payload, params=params) def live_url_scan( - self, url: str, platform: str | None = None, os: str | None = None, browser: str | None = None, region: str | None = None + self, + url: str, + platform: str | None = None, + os: str | None = None, + browser: str | None = None, + region: str | None = None, ) -> dict[str, Any]: """ Perform a live scan of a URL to get hosting metadata. @@ -2239,7 +2431,8 @@ def get_future_attack_indicators(self, feed_uuid: str, page_no: int = 1, page_si Returns: Dict[str, Any]: Response containing future attack indicators. """ - params = {"feed_uuids": feed_uuid, "page": page_no, "size": page_size} + + params = {"source_uuids": feed_uuid, "page": page_no, "limit": page_size} query_string = urlencode(params) url = self._base_url.replace("/api/v1/merge-api", "") + f"/api/v2/iocs/threat-ranking/?{query_string}" @@ -2261,7 +2454,6 @@ def screenshot_url(self, url: str) -> dict[str, Any]: remove_nulls_from_dictionary(params) response = self._http_request(method="GET", url_suffix=endpoint, params=params) - if response.get("error"): return {"error": f"Failed to get screenshot: {response['error']}"} @@ -2365,9 +2557,9 @@ def get_job_status_command(client: Client, args: dict) -> CommandResults: @metadata_collector.command( command_name="silentpush-get-nameserver-reputation", inputs_list=NAMESERVER_REPUTATION_INPUTS, - outputs_prefix="SilentPush.SubnetReputation", + outputs_prefix="SilentPush.NameserverReputation", outputs_list=NAMESERVER_REPUTATION_OUTPUTS, - description="This command retrieve historical reputation data for a specified nameserver, " + description="This command retrieves historical reputation data for a specified nameserver," "including reputation scores and optional detailed calculation information.", ) def get_nameserver_reputation_command(client: Client, args: dict) -> CommandResults: @@ -2382,24 +2574,39 @@ def get_nameserver_reputation_command(client: Client, args: dict) -> CommandResu CommandResults: The command results containing nameserver reputation data. """ nameserver = args.get("nameserver") - explain = argToBoolean(args.get("explain", False)) + explain = argToBoolean(args.get("explain", "false")) limit = arg_to_number(args.get("limit")) if not nameserver: raise ValueError("Nameserver is required.") - # Fetch reputation data reputation_data = client.get_nameserver_reputation(nameserver, explain, limit) - # Prepare the readable output - if reputation_data: + if not isinstance(reputation_data, list): + demisto.error(f"Expected list, got: {type(reputation_data)}") + reputation_data = [] + + for item in reputation_data: + date_val = item.get("date") + if isinstance(date_val, int): + try: + parsed_date = datetime.strptime(str(date_val), "%Y%m%d").date() + item["date"] = parsed_date.isoformat() + except ValueError as e: + demisto.error(f"Failed to parse date {date_val}: {e}") + + if reputation_data and all(isinstance(item, dict) for item in reputation_data): + all_headers = set() + for item in reputation_data: + all_headers.update(item.keys()) + readable_output = tableToMarkdown( - f"Nameserver Reputation for {nameserver}", reputation_data, headers=list(reputation_data[0].keys()), removeNull=True + f"Nameserver Reputation for {nameserver}", reputation_data, headers=sorted(all_headers), removeNull=True ) else: - readable_output = f"No reputation history found for nameserver: {nameserver}" + readable_output = f"No valid reputation history found for nameserver: {nameserver}" + reputation_data = [] - # Return command results return CommandResults( outputs_prefix="SilentPush.NameserverReputation", outputs_key_field="ns_server", @@ -2412,7 +2619,7 @@ def get_nameserver_reputation_command(client: Client, args: dict) -> CommandResu @metadata_collector.command( command_name="silentpush-get-subnet-reputation", inputs_list=SUBNET_REPUTATION_INPUTS, - outputs_prefix="SilentPush.SubnetReputation", + outputs_prefix="SilentPush.NameserverReputation", outputs_list=SUBNET_REPUTATION_OUTPUTS, description="This command retrieves the reputation history for a specific subnet.", ) @@ -2664,16 +2871,21 @@ def list_domain_infratags_command(client: Client, args: dict) -> CommandResults: """ domains = argToList(args.get("domains", "")) cluster = argToBoolean(args.get("cluster", False)) - mode = args.get("mode", "live") + mode = args.get("mode", "live").lower() match = args.get("match", "self") - as_of = args.get("as_of", None) - origin_uid = args.get("origin_uid", None) + as_of = args.get("as_of") + origin_uid = args.get("origin_uid") use_get = argToBoolean(args.get("use_get", False)) if not domains and not use_get: raise ValueError('"domains" argument is required when using POST.') - raw_response = client.list_domain_infratags(domains, cluster, mode, match, as_of, origin_uid) + raw_response = client.list_domain_infratags(domains, cluster, mode=mode, match=match, as_of=as_of, origin_uid=origin_uid) + + response_mode = raw_response.get("response", {}).get("mode", "").lower() + if response_mode and response_mode != mode: + raise ValueError(f"Expected mode '{mode}' but got '{response_mode}'") + infratags = raw_response.get("response", {}).get("infratags", []) tag_clusters = raw_response.get("response", {}).get("tag_clusters", []) @@ -2774,7 +2986,7 @@ def format_domain_information(response: dict[str, Any], fetch_risk_score: bool, whois_info = domain_data.get("whois_info", {}) if whois_info and isinstance(whois_info, dict): if "error" in whois_info: - markdown.append(f'WHOIS Error: {whois_info["error"]}') + markdown.append(f"WHOIS Error: {whois_info['error']}") else: markdown.append(tableToMarkdown("WHOIS Information", [whois_info])) @@ -3085,8 +3297,14 @@ def get_asn_reputation_command(client: Client, args: dict) -> CommandResults: CommandResults: Formatted command results for XSOAR """ asn = args.get("asn") - limit = arg_to_number(args.get("limit", None)) - explain = argToBoolean(args.get("explain", False)) + if not asn: + raise ValueError("ASN is required.") + try: + asn = int(asn) + except ValueError: + raise ValueError("Invalid ASN number") + limit = arg_to_number(args.get("limit")) + explain = argToBoolean(args.get("explain", "false")) if not asn: raise ValueError("ASN is required.") @@ -3095,7 +3313,13 @@ def get_asn_reputation_command(client: Client, args: dict) -> CommandResults: asn_reputation = extract_and_sort_asn_reputation(raw_response) if not asn_reputation: - return generate_no_reputation_response(asn, raw_response) + return CommandResults( + readable_output=f"No reputation data found for ASN {asn}.", + outputs_prefix="SilentPush.ASNReputation", + outputs_key_field="asn", + outputs=[], + raw_response=raw_response, + ) data_for_table = prepare_asn_reputation_table(asn_reputation, explain) readable_output = tableToMarkdown(f"ASN Reputation for {asn}", data_for_table, headers=get_table_headers(explain)) @@ -3121,17 +3345,14 @@ def extract_and_sort_asn_reputation(raw_response: dict) -> list: """ response_data = raw_response.get("response", {}) - # Handle unexpected format gracefully if not isinstance(response_data, dict): response_data = {"asn_reputation": response_data} asn_reputation = response_data.get("asn_reputation") or response_data.get("asn_reputation_history", []) - # Normalize to list if isinstance(asn_reputation, dict): asn_reputation = [asn_reputation] elif not isinstance(asn_reputation, list): - # If it's something else (e.g., a string), fallback to an empty list asn_reputation = [] return sorted(asn_reputation, key=lambda x: x.get("date", ""), reverse=True) @@ -3225,30 +3446,46 @@ def get_asn_takedown_reputation_command(client, args): """ asn = args.get("asn") if not asn: - raise ValueError("ASN is a required parameter") + raise ValueError("ASN is a required parameter.") - explain = argToBoolean(args.get("explain", False)) - limit = arg_to_number(args.get("limit")) + try: + explain = argToBoolean(args.get("explain", False)) + limit = arg_to_number(args.get("limit")) + except Exception as e: + raise ValueError(f"Invalid argument: {e}") - response = client.get_asn_takedown_reputation(asn, explain, limit) + try: + response = client.get_asn_takedown_reputation(asn, explain, limit) + except Exception as e: + raise DemistoException(f"API call failed: {str(e)}") - if isinstance(response, dict): - table_data = [response] - elif isinstance(response, str): - table_data = [{"message": response}] - else: - table_data = [{"error": "Invalid response format"}] + takedown_history = response.get("takedown_reputation") + + if not takedown_history: + return CommandResults( + readable_output=f"No takedown reputation history found for ASN: {asn}", + outputs_prefix="SilentPush.ASNTakedownReputation", + outputs_key_field="asn", + outputs=takedown_history, + raw_response=response, + ) - # Explicitly specify headers from keys of first dict - headers = list(table_data[0].keys()) if table_data else ["message"] + if isinstance(takedown_history.get("asn_allocation_date"), int): + try: + takedown_history["asn_allocation_date"] = ( + datetime.strptime(str(takedown_history["asn_allocation_date"]), "%Y%m%d").date().isoformat() + ) + except ValueError: + demisto.debug(f"Failed to format date: {takedown_history.get('asn_allocation_date')}") - readable_output = tableToMarkdown(f"Takedown Reputation for ASN {asn}", table_data, headers=headers, removeNull=True) + readable_output = tableToMarkdown(f"Takedown Reputation for ASN {asn}", takedown_history, removeNull=True) return CommandResults( readable_output=readable_output, outputs_prefix="SilentPush.ASNTakedownReputation", outputs_key_field="asn", - outputs=table_data, + outputs=takedown_history, + raw_response=response, ) @@ -3271,7 +3508,6 @@ def get_ipv4_reputation_command(client: Client, args: dict[str, Any]) -> Command CommandResults: The results of the command including the IPv4 reputation data. """ ipv4 = args.get("ipv4") - if not ipv4: raise DemistoException("IPv4 address is required") @@ -3282,10 +3518,8 @@ def get_ipv4_reputation_command(client: Client, args: dict[str, Any]) -> Command raw_response = client.get_ipv4_reputation(ipv4, explain, limit) - # Defensive extraction - history = raw_response.get("response", {}).get("ip_reputation_history", {}) - - if not history or history.get("error") == "Not found": + history = raw_response.get("ip_reputation_history", {}) + if not history: return CommandResults( readable_output=f"No reputation data found for IPv4: {ipv4}", outputs_prefix="SilentPush.IPv4Reputation", @@ -3293,27 +3527,11 @@ def get_ipv4_reputation_command(client: Client, args: dict[str, Any]) -> Command outputs={"ip": ipv4}, raw_response=raw_response, ) - - # Construct result - reputation_data = { - "ip": history.get("ipv4", ipv4), - "date": history.get("date"), - "reputation_score": history.get("ip_reputation"), - } - - explain_data = history.get("ip_reputation_explain", {}) - if explain_data: - reputation_data["ip_reputation_explain"] = { - "ip_density": explain_data.get("ip_density"), - "names_num_listed": explain_data.get("names_num_listed"), - } - - readable_output = tableToMarkdown(f"IPv4 Reputation Information for {ipv4}", [reputation_data]) - + readable_output = tableToMarkdown(f"IPv4 Reputation Information for {ipv4}", history) return CommandResults( outputs_prefix="SilentPush.IPv4Reputation", outputs_key_field="ip", - outputs=reputation_data, + outputs=history, readable_output=readable_output, raw_response=raw_response, ) @@ -3646,11 +3864,26 @@ def screenshot_url_command(client: Client, args: dict[str, Any]) -> CommandResul if result.get("error"): raise Exception(result.get("error")) - image_response = generic_http_request("GET", result["screenshot_url"], url_suffix="", resp_type="response") - if image_response.status_code != 200: - return {"error": f"Failed to download screenshot image: HTTP {image_response.status_code}"} + if not result.get("screenshot_url"): + raise ValueError("screenshot_url is missing from API response.") + + screenshot_url = result["screenshot_url"] + parsed_url = urlparse(screenshot_url) + + if not parsed_url.scheme or not parsed_url.netloc: + raise ValueError(f"Invalid screenshot URL format: {screenshot_url}") + + server_url = f"{parsed_url.scheme}://{parsed_url.netloc}" + url_suffix = parsed_url.path + if parsed_url.query: + url_suffix += f"?{parsed_url.query}" + + image_response = generic_http_request(method="GET", server_url=server_url, url_suffix=url_suffix, resp_type="response") + + if not image_response or image_response.status_code != 200: + return {"error": f"Failed to download screenshot image: HTTP {getattr(image_response, 'status_code', 'No response')}"} - filename = f"{url.split('://')[1].split('/')[0]}_screenshot.jpg" + filename = f"{urlparse(url).netloc}_screenshot.jpg" readable_output = ( f"### Screenshot captured for {url}\n" @@ -3662,13 +3895,12 @@ def screenshot_url_command(client: Client, args: dict[str, Any]) -> CommandResul result_data = { "url": url, "status": "success", - "status_code": result["status_code"], + "status_code": result.get("status_code"), "screenshot_url": result["screenshot_url"], - "file_name": "filename", + "file_name": filename, } remove_nulls_from_dictionary(result_data) - # Download link of the image return_results(fileResult(filename, image_response.content)) return CommandResults( diff --git a/Packs/SilentPush/Integrations/SilentPush/SilentPush.yml b/Packs/SilentPush/Integrations/SilentPush/SilentPush.yml index f4ef458acc96..11c8802f3fdc 100644 --- a/Packs/SilentPush/Integrations/SilentPush/SilentPush.yml +++ b/Packs/SilentPush/Integrations/SilentPush/SilentPush.yml @@ -1355,22 +1355,22 @@ script: secret: false default: false outputs: - - contextPath: SilentPush.SubnetReputation.nameserver + - contextPath: SilentPush.NameserverReputation.nameserver description: The nameserver associated with the reputation history entry. type: Number - - contextPath: SilentPush.SubnetReputation.reputation_data.date + - contextPath: SilentPush.NameserverReputation.reputation_data.date description: Date of the reputation history entry (in YYYYMMDD format). type: Number - - contextPath: SilentPush.SubnetReputation.reputation_data.ns_server + - contextPath: SilentPush.NameserverReputation.reputation_data.ns_server description: Name of the nameserver associated with the reputation history entry. type: String - - contextPath: SilentPush.SubnetReputation.reputation_data.ns_server_reputation + - contextPath: SilentPush.NameserverReputation.reputation_data.ns_server_reputation description: Reputation score of the nameserver on the specified date. type: Number - - contextPath: SilentPush.SubnetReputation.reputation_data.ns_server_reputation_explain.ns_server_domain_density + - contextPath: SilentPush.NameserverReputation.reputation_data.ns_server_reputation_explain.ns_server_domain_density description: Number of domains associated with the nameserver. type: Number - - contextPath: SilentPush.SubnetReputation.reputation_data.ns_server_reputation_explain.ns_server_domains_listed + - contextPath: SilentPush.NameserverReputation.reputation_data.ns_server_reputation_explain.ns_server_domains_listed description: Number of domains listed in reputation databases. type: Number - deprecated: false diff --git a/Packs/SilentPush/Integrations/SilentPush/SilentPush_test.py b/Packs/SilentPush/Integrations/SilentPush/SilentPush_test.py index dce16b1ff959..e90a677d83cf 100644 --- a/Packs/SilentPush/Integrations/SilentPush/SilentPush_test.py +++ b/Packs/SilentPush/Integrations/SilentPush/SilentPush_test.py @@ -33,7 +33,6 @@ density_lookup_command, ) from CommonServerPython import DemistoException -from requests.models import Response def util_load_json(path): @@ -93,54 +92,59 @@ def test_get_job_status_command_no_status_found(mock_client): def test_get_nameserver_reputation_command_success(mock_client, mocker): - # Mock arguments - args = {"nameserver": "example.com", "explain": "true", "limit": "10"} + args = {"nameserver": "example.com", "explain": "true", "limit": 10} - # Mock response from client - mock_response = [{"ns_server": "example.com", "reputation": "good", "details": "No issues found"}] - mock_client.get_nameserver_reputation.return_value = mock_response + mock_response = { + "response": { + "ns_server_reputation_history": [ + {"ns_server": "example.com", "reputation": "good", "details": "No issues found", "date": 20240101} + ] + } + } - # Mock tableToMarkdown + mock_client.get_nameserver_reputation.return_value = mock_response["response"]["ns_server_reputation_history"] mocker.patch("SilentPush.tableToMarkdown", return_value="Mocked Markdown Table") - # Call the function result = get_nameserver_reputation_command(mock_client, args) - # Assertions + mock_client.get_nameserver_reputation.assert_called_once_with("example.com", True, 10) assert isinstance(result, CommandResults) assert result.outputs_prefix == "SilentPush.NameserverReputation" assert result.outputs_key_field == "ns_server" assert result.outputs["nameserver"] == "example.com" - assert result.outputs["reputation_data"] == mock_response + assert result.outputs["reputation_data"][0]["date"] == "2024-01-01" assert result.readable_output == "Mocked Markdown Table" def test_get_nameserver_reputation_command_no_nameserver(mock_client): - # Mock arguments without nameserver - args = {} - - # Call the function and expect ValueError + args = {"explain": "true"} with pytest.raises(ValueError, match="Nameserver is required."): get_nameserver_reputation_command(mock_client, args) def test_get_nameserver_reputation_command_no_data(mock_client, mocker): - # Mock arguments - args = {"nameserver": "example.com", "explain": "false", "limit": "5"} - - # Mock response from client + args = {"nameserver": "example.com"} mock_client.get_nameserver_reputation.return_value = [] - # Call the function result = get_nameserver_reputation_command(mock_client, args) - - # Assertions assert isinstance(result, CommandResults) - assert result.outputs_prefix == "SilentPush.NameserverReputation" - assert result.outputs_key_field == "ns_server" - assert result.outputs["nameserver"] == "example.com" assert result.outputs["reputation_data"] == [] - assert result.readable_output == "No reputation history found for nameserver: example.com" + assert result.readable_output == "No valid reputation history found for nameserver: example.com" + + +def test_get_nameserver_reputation_command_date_formatting(mock_client, mocker): + args = {"nameserver": "example.com"} + mock_response = [ + {"ns_server": "example.com", "date": 20240215}, + {"ns_server": "example.com", "date": "not_a_date"}, + ] + + mock_client.get_nameserver_reputation.return_value = mock_response + mocker.patch("SilentPush.tableToMarkdown", return_value="Mocked Table") + + result = get_nameserver_reputation_command(mock_client, args) + assert result.outputs["reputation_data"][0]["date"] == "2024-02-15" + assert result.outputs["reputation_data"][1]["date"] == "not_a_date" def test_get_subnet_reputation_command_success(mock_client, mocker): @@ -259,7 +263,6 @@ def test_get_asns_for_domain_command_no_data(mock_client, mocker): def test_list_domain_infratags_command_success(mock_client, mocker): - # Mock arguments args = { "domains": "example.com,example.org", "cluster": "true", @@ -267,60 +270,87 @@ def test_list_domain_infratags_command_success(mock_client, mocker): "match": "self", "as_of": "2023-01-01", "origin_uid": "12345", - "use_get": "false", } - # Mock response from client mock_response = { "response": { - "infratags": [{"domain": "example.com", "tag": "tag1"}, {"domain": "example.org", "tag": "tag2"}], - "tag_clusters": [{"cluster_name": "Cluster1", "tags": ["tag1", "tag2"]}], + "mode": "live", + "infratags": [ + {"domain": "example.com", "tags": ["tag1", "tag2"]}, + {"domain": "example.org", "tags": ["tag3", "tag4"]}, + ], + "tag_clusters": [ + {"cluster_name": "Cluster1", "tags": ["tag1", "tag2"]}, + {"cluster_name": "Cluster2", "tags": ["tag3", "tag4"]}, + ], } } - mock_client.list_domain_infratags.return_value = mock_response - # Mock tableToMarkdown and format_tag_clusters - mocker.patch("SilentPush.tableToMarkdown", return_value="Mocked Markdown Table") - mocker.patch("SilentPush.format_tag_clusters", return_value="\nMocked Cluster Details") + mock_client.list_domain_infratags.return_value = mock_response + mocker.patch("SilentPush.tableToMarkdown", return_value="Mocked Table") - # Call the function result = list_domain_infratags_command(mock_client, args) - # Assertions + mock_client.list_domain_infratags.assert_called_once_with( + ["example.com", "example.org"], True, mode="live", match="self", as_of="2023-01-01", origin_uid="12345" + ) + assert isinstance(result, CommandResults) assert result.outputs_prefix == "SilentPush.InfraTags" assert result.outputs_key_field == "domain" assert result.outputs == mock_response - assert "Mocked Markdown Table" in result.readable_output - assert "Mocked Cluster Details" in result.readable_output + assert "Mocked Table" in result.readable_output -def test_list_domain_infratags_command_no_domains(mock_client): - # Mock arguments without domains - args = {"use_get": "false"} +def test_list_domain_infratags_command_non_live_mode(mock_client): + args = {"domains": "example.com", "mode": "live"} + mock_response = {"response": {"mode": "historical"}} + mock_client.list_domain_infratags.return_value = mock_response + + with pytest.raises(ValueError, match="Expected mode 'live' but got 'historical'"): + list_domain_infratags_command(mock_client, args) + + +def test_list_domain_infratags_command_empty_domains(mock_client): + # Test with empty domains and use_get=False + args = {"domains": "", "use_get": "false"} - # Call the function and expect ValueError with pytest.raises(ValueError, match='"domains" argument is required when using POST.'): list_domain_infratags_command(mock_client, args) -def test_list_domain_infratags_command_no_data(mock_client, mocker): - # Mock arguments - args = {"domains": "example.com", "cluster": "false", "use_get": "true"} +def test_list_domain_infratags_command_use_get(mock_client, mocker): + # Test with use_get=True and no domains + args = {"use_get": "true"} + mock_response = {"response": {"mode": "live", "infratags": []}} + mock_client.list_domain_infratags.return_value = mock_response + mocker.patch("CommonServerPython.tableToMarkdown", return_value="Empty Table") - # Mock response from client - mock_response = {"response": {"infratags": [], "tag_clusters": []}} + result = list_domain_infratags_command(mock_client, args) + assert isinstance(result, CommandResults) + assert result.outputs == mock_response + + +def test_list_domain_infratags_command_empty_response(mock_client, mocker): + # Test with empty response + args = {"domains": "example.com"} + mock_response = {"response": {"mode": "live", "infratags": []}} mock_client.list_domain_infratags.return_value = mock_response - # Call the function result = list_domain_infratags_command(mock_client, args) + assert isinstance(result, CommandResults) + assert not result.outputs.get("response", {}).get("infratags") - # Assertions + +def test_list_domain_infratags_command_invalid_response(mock_client): + # Test with invalid response structure + args = {"domains": "example.com"} + mock_response = {"invalid": "response"} + mock_client.list_domain_infratags.return_value = mock_response + + result = list_domain_infratags_command(mock_client, args) assert isinstance(result, CommandResults) - assert result.outputs_prefix == "SilentPush.InfraTags" - assert result.outputs_key_field == "domain" - assert result.outputs == mock_response - assert result.readable_output.strip() == "### Domain Infratags\n**No entries.**" + assert not result.outputs.get("response", {}).get("infratags") def test_list_domain_information_command_success(mock_client, mocker): @@ -394,7 +424,6 @@ def test_get_ipv4_reputation_command_no_ipv4(mock_client): def test_get_future_attack_indicators_command_success(mock_client, mocker): - # Mock arguments args = {"feed_uuid": "test-feed-uuid", "page_no": "1", "page_size": "10"} # Mock response from client @@ -428,13 +457,15 @@ def test_get_future_attack_indicators_command_no_feed_uuid(mock_client): def test_get_future_attack_indicators_command_no_data(mock_client, mocker): - # Mock arguments args = {"feed_uuid": "test-feed-uuid", "page_no": "1", "page_size": "10"} # Mock response from client mock_response = [] mock_client.get_future_attack_indicators.return_value = mock_response + # Mock tableToMarkdown + mocker.patch("SilentPush.tableToMarkdown", return_value="### SilentPush Future Attack Indicators\n**No entries.**") + # Call the function result = get_future_attack_indicators_command(mock_client, args) @@ -456,7 +487,10 @@ def test_list_ip_information_command_success(mock_client, mocker): # Mock gather_ip_information mocker.patch( "SilentPush.gather_ip_information", - side_effect=[[{"ip": "192.168.1.1", "info": "IPv4 info"}], [{"ip": "2001:db8::ff00:42:8329", "info": "IPv6 info"}]], + side_effect=[ + [{"ip": "192.168.1.1", "info": "IPv4 info"}], + [{"ip": "2001:db8::ff00:42:8329", "info": "IPv6 info"}], + ], ) # Mock tableToMarkdown @@ -469,7 +503,10 @@ def test_list_ip_information_command_success(mock_client, mocker): assert isinstance(result, CommandResults) assert result.outputs_prefix == "SilentPush.IPInformation" assert result.outputs_key_field == "ip" - assert result.outputs == [{"ip": "192.168.1.1", "info": "IPv4 info"}, {"ip": "2001:db8::ff00:42:8329", "info": "IPv6 info"}] + assert result.outputs == [ + {"ip": "192.168.1.1", "info": "IPv4 info"}, + {"ip": "2001:db8::ff00:42:8329", "info": "IPv6 info"}, + ] assert result.readable_output == "Mocked Markdown Table" @@ -509,151 +546,107 @@ def test_list_ip_information_command_no_data(mock_client, mocker): assert result.readable_output == "No information found for IPs: 192.168.1.1" -def test_get_ipv4_reputation_command_success_second(mock_client, mocker): - # Mock arguments - args = {"ipv4": "192.168.1.1", "explain": "true", "limit": "1"} - - # Mock validate_ip - mocker.patch("SilentPush.validate_ip", return_value=True) +def test_get_asn_takedown_reputation_command_success(mock_client, mocker): + args = {"asn": "13335", "limit": "10", "explain": "false"} - # Mock response from client mock_response = { - "response": { - "ip_reputation_history": { - "ip": "192.168.1.1", - "date": "2023-01-01", - "ip_reputation": 85, - "ip_reputation_explain": {"ip_density": 0.5, "names_num_listed": 10}, - } + "takedown_reputation": { + "asn": 13335, + "asn_allocation_age": 4014, + "asn_allocation_date": 20100714, + "asn_takedown_reputation": 0, + "asname": "CLOUDFLARENET, US", } } - mock_client.get_ipv4_reputation.return_value = mock_response - - # Mock tableToMarkdown - mocker.patch("SilentPush.tableToMarkdown", return_value="Mocked Markdown Table") - - # Call the function - result = get_ipv4_reputation_command(mock_client, args) - - # Assertions - assert isinstance(result, CommandResults) - assert result.outputs_prefix == "SilentPush.IPv4Reputation" - assert result.outputs_key_field == "ip" - assert result.outputs["ip"] == "192.168.1.1" - assert result.outputs["reputation_score"] == 85 - assert result.outputs["ip_reputation_explain"] == {"ip_density": 0.5, "names_num_listed": 10} - assert result.readable_output == "Mocked Markdown Table" - - -def test_get_ipv4_reputation_command_no_data_second(mock_client, mocker): - # Mock arguments - args = {"ipv4": "192.168.1.1", "explain": "false", "limit": "1"} - - # Mock validate_ip - mocker.patch("SilentPush.validate_ip", return_value=True) - - # Mock response from client - mock_response = {} - mock_client.get_ipv4_reputation.return_value = mock_response - - # Call the function - result = get_ipv4_reputation_command(mock_client, args) - # Assertions - assert isinstance(result, CommandResults) - assert result.outputs_prefix == "SilentPush.IPv4Reputation" - assert result.outputs_key_field == "ip" - assert result.outputs["ip"] == "192.168.1.1" - assert result.readable_output == "No reputation data found for IPv4: 192.168.1.1" - - -def test_get_asn_takedown_reputation_command_success(mock_client, mocker): - # Mock arguments - args = {"asn": "12345", "limit": "10", "explain": "true"} - - # Mock response from client - mock_response = {"asn": "12345", "reputation_score": 85, "details": "ASN is associated with malicious activity"} mock_client.get_asn_takedown_reputation.return_value = mock_response - # Mock tableToMarkdown mocker.patch("SilentPush.tableToMarkdown", return_value="Mocked Markdown Table") - # Call the function result = get_asn_takedown_reputation_command(mock_client, args) - # Assertions assert isinstance(result, CommandResults) assert result.outputs_prefix == "SilentPush.ASNTakedownReputation" assert result.outputs_key_field == "asn" - assert result.outputs[0] == mock_response - assert result.readable_output == "Mocked Markdown Table" + assert result.outputs == { + "asn": 13335, + "asn_allocation_age": 4014, + "asn_allocation_date": "2010-07-14", # formatted date + "asn_takedown_reputation": 0, + "asname": "CLOUDFLARENET, US", + } def test_get_asn_takedown_reputation_command_no_asn(mock_client): - # Mock arguments without asn args = {} - - # Call the function and expect ValueError - with pytest.raises(ValueError, match="ASN is a required parameter"): + with pytest.raises(ValueError, match="ASN is a required parameter."): get_asn_takedown_reputation_command(mock_client, args) def test_get_asn_takedown_reputation_command_invalid_limit(mock_client): - # Mock arguments with invalid limit args = {"asn": "12345", "limit": "invalid"} - - # Call the function and expect ValueError - with pytest.raises(ValueError, match='"invalid" is not a valid number'): + with pytest.raises(ValueError, match="Invalid argument:"): get_asn_takedown_reputation_command(mock_client, args) -def test_get_asn_takedown_reputation_command_no_data(mock_client, mocker): - # Mock arguments +def test_get_asn_takedown_reputation_command_no_data(mock_client): args = {"asn": "12345", "limit": "10", "explain": "false"} - # Mock response from client - mock_client.get_asn_takedown_reputation.return_value = None + mock_response = {"asn": "12345"} + mock_client.get_asn_takedown_reputation.return_value = mock_response - # Call the function result = get_asn_takedown_reputation_command(mock_client, args) - # Assertions assert isinstance(result, CommandResults) assert result.outputs_prefix == "SilentPush.ASNTakedownReputation" - assert result.outputs == [{"error": "Invalid response format"}] - assert ( - result.readable_output - == """### Takedown Reputation for ASN 12345 -|error| -|---| -| Invalid response format | -""" - ) + assert result.outputs_key_field == "asn" + assert result.outputs is None + assert "No takedown reputation history found for ASN: 12345" in result.readable_output def test_get_asn_reputation_command_success(mock_client, mocker): - # Mock arguments args = {"asn": "12345", "limit": "10", "explain": "true"} - # Mock response from client - mock_response = [{"asn": "12345", "reputation_score": 85, "details": "ASN is associated with suspicious activity"}] - mock_client.get_asn_reputation.return_value = mock_response + mock_raw_response = { + "asn_reputation_history": [ + { + "asn": 12345, + "reputation_score": 85, + "timestamp": "2024-01-22T10:00:00Z", + "explanation": {"malicious_activity": 0.7, "spam_activity": 0.3}, + } + ] + } + + mock_processed_response = [ + { + "asn": 12345, + "reputation_score": 85, + "timestamp": "2024-01-22T10:00:00Z", + "explanation": {"malicious_activity": 0.7, "spam_activity": 0.3}, + } + ] + + mock_client.get_asn_reputation.return_value = mock_raw_response - # Mock helper functions - mocker.patch("SilentPush.extract_and_sort_asn_reputation", return_value=mock_response) - mocker.patch("SilentPush.prepare_asn_reputation_table", return_value=mock_response) - mocker.patch("SilentPush.get_table_headers", return_value=["asn", "reputation_score", "details"]) + mocker.patch("SilentPush.extract_and_sort_asn_reputation", return_value=mock_processed_response) + mocker.patch("SilentPush.prepare_asn_reputation_table", return_value=mock_processed_response[0]) mocker.patch("SilentPush.tableToMarkdown", return_value="Mocked Markdown Table") - # Call the function result = get_asn_reputation_command(mock_client, args) - # Assertions + # Check that client was called with correct params + called_args = mock_client.get_asn_reputation.call_args[0] + assert called_args[0] == 12345 + assert called_args[1] == 10 + assert called_args[2] is True + assert isinstance(result, CommandResults) assert result.outputs_prefix == "SilentPush.ASNReputation" assert result.outputs_key_field == "asn" - assert result.outputs == mock_response + assert result.outputs == mock_processed_response assert result.readable_output == "Mocked Markdown Table" + assert result.raw_response == mock_raw_response def test_get_asn_reputation_command_no_asn(mock_client): @@ -665,31 +658,55 @@ def test_get_asn_reputation_command_no_asn(mock_client): get_asn_reputation_command(mock_client, args) +def test_get_asn_reputation_command_invalid_asn(mock_client): + # Mock arguments with invalid ASN + args = {"asn": "invalid"} + + # Call the function and expect ValueError + with pytest.raises(ValueError, match="Invalid ASN number"): + get_asn_reputation_command(mock_client, args) + + def test_get_asn_reputation_command_no_data(mock_client, mocker): - # Mock arguments - args = {"asn": "12345", "limit": "10", "explain": "false"} + args = {"asn": "12345"} - # Mock response from client - mock_response = [] - mock_client.get_asn_reputation.return_value = mock_response + mock_raw_response = {"asn_reputation_history": []} + mock_client.get_asn_reputation.return_value = mock_raw_response - # Mock helper functions mocker.patch("SilentPush.extract_and_sort_asn_reputation", return_value=[]) - mocker.patch( - "SilentPush.generate_no_reputation_response", - return_value=CommandResults( - readable_output="No reputation data found for ASN 12345", outputs_prefix="SilentPush.ASNReputation", outputs=None - ), - ) - # Call the function result = get_asn_reputation_command(mock_client, args) - # Assertions + # Check client call values using call_args + called_args = mock_client.get_asn_reputation.call_args[0] + assert called_args[0] == 12345 + assert called_args[1] is None + assert called_args[2] is False + assert isinstance(result, CommandResults) assert result.outputs_prefix == "SilentPush.ASNReputation" - assert result.outputs is None - assert result.readable_output == "No reputation data found for ASN 12345" + assert result.outputs == [] + assert "No reputation data found for ASN 12345" in result.readable_output + assert result.raw_response == mock_raw_response + + +def test_get_asn_reputation_command_with_limit(mock_client, mocker): + args = {"asn": "12345", "limit": "5"} + + mock_raw_response = {"asn_reputation_history": []} + mock_client.get_asn_reputation.return_value = mock_raw_response + + mocker.patch("SilentPush.extract_and_sort_asn_reputation", return_value=[]) + + result = get_asn_reputation_command(mock_client, args) + + called_args = mock_client.get_asn_reputation.call_args[0] + assert called_args[0] == 12345 + assert called_args[1] == 5 + assert called_args[2] is False + + assert isinstance(result, CommandResults) + assert result.outputs == [] def test_get_enrichment_data_command_success(mock_client, mocker): @@ -751,8 +768,18 @@ def test_get_domain_certificates_command_success(mock_client, mocker): mock_response = { "response": { "domain_certificates": [ - {"certificate_id": "123", "issuer": "Example Issuer", "valid_from": "2023-01-01", "valid_to": "2024-01-01"}, - {"certificate_id": "456", "issuer": "Another Issuer", "valid_from": "2022-01-01", "valid_to": "2023-01-01"}, + { + "certificate_id": "123", + "issuer": "Example Issuer", + "valid_from": "2023-01-01", + "valid_to": "2024-01-01", + }, + { + "certificate_id": "456", + "issuer": "Another Issuer", + "valid_from": "2022-01-01", + "valid_to": "2023-01-01", + }, ], "metadata": {"total": 2}, } @@ -830,74 +857,77 @@ def test_get_domain_certificates_command_job_status(mock_client, mocker): def test_screenshot_url_command_success(mock_client, mocker): - # Mock arguments args = {"url": "https://example.com"} - # Mock response from client - mock_response = {"screenshot_url": "https://example.com/screenshot.jpg", "status_code": 200} + # Mock the client response + mock_response = {"screenshot_url": "https://storage.com/path/screenshot.jpg", "status_code": 200} mock_client.screenshot_url.return_value = mock_response - # Mock requests.get - mock_image_response = mocker.Mock(spec=Response) + # Properly patch where the function is used, not defined + mock_image_response = mocker.Mock() mock_image_response.status_code = 200 mock_image_response.content = b"image content" - mocker.patch("requests.Session.request", return_value=mock_image_response) + mocker.patch("SilentPush.generic_http_request", return_value=mock_image_response) - # Mock fileResult - mocker.patch("SilentPush.fileResult", return_value={"Type": 3, "FileID": "123", "File": "example_screenshot.jpg"}) + mock_file_result = mocker.patch("SilentPush.fileResult", return_value={"Type": 3, "FileID": "123"}) + mock_return_results = mocker.patch("SilentPush.return_results") - # Call the function + # Run command result = screenshot_url_command(mock_client, args) - # Assertions + # Validate assert isinstance(result, CommandResults) - assert result.outputs_prefix == "SilentPush.Screenshot" - assert result.outputs_key_field == "url" assert result.outputs["url"] == "https://example.com" assert result.outputs["status"] == "success" - assert result.outputs["screenshot_url"] == "https://example.com/screenshot.jpg" - assert "Screenshot captured for https://example.com" in result.readable_output + assert result.outputs["screenshot_url"] == mock_response["screenshot_url"] + assert result.outputs["file_name"] == "example.com_screenshot.jpg" + mock_file_result.assert_called_once_with("example.com_screenshot.jpg", mock_image_response.content) + mock_return_results.assert_called_once() -def test_screenshot_url_command_no_url(mock_client): - # Mock arguments without URL - args = {} - # Call the function and expect ValueError +def test_screenshot_url_command_no_url(mock_client): + # Test with empty args with pytest.raises(ValueError, match="URL is required"): + screenshot_url_command(mock_client, {}) + + +def test_screenshot_url_command_missing_screenshot_url(mock_client): + args = {"url": "https://example.com"} + mock_client.screenshot_url.return_value = {"status": "success"} + + with pytest.raises(ValueError, match="screenshot_url is missing from API response"): screenshot_url_command(mock_client, args) def test_screenshot_url_command_error_from_api(mock_client): - # Mock arguments args = {"url": "https://example.com"} + mock_client.screenshot_url.return_value = {"error": "API Error"} - # Mock response from client with error - mock_response = {"error": "Invalid URL"} - mock_client.screenshot_url.return_value = mock_response + with pytest.raises(Exception, match="API Error"): + screenshot_url_command(mock_client, args) + + +def test_screenshot_url_command_invalid_screenshot_url(mock_client): + args = {"url": "https://example.com"} + mock_client.screenshot_url.return_value = {"screenshot_url": "invalid_url"} - # Call the function and expect Exception - with pytest.raises(Exception, match="Invalid URL"): + with pytest.raises(ValueError, match="Invalid screenshot URL format"): screenshot_url_command(mock_client, args) def test_screenshot_url_command_failed_image_download(mock_client, mocker): - # Mock arguments args = {"url": "https://example.com"} - - # Mock response from client - mock_response = {"screenshot_url": "https://example.com/screenshot.jpg", "status_code": 200} + mock_response = {"screenshot_url": "https://storage.com/path/screenshot.jpg"} mock_client.screenshot_url.return_value = mock_response - # Mock requests.get with failed status code - mock_image_response = mocker.Mock(spec=Response) + mock_image_response = mocker.Mock() mock_image_response.status_code = 404 - mocker.patch("requests.Session.request", return_value=mock_image_response) - # Call the function - result = screenshot_url_command(mock_client, args) + mocker.patch("SilentPush.generic_http_request", return_value=mock_image_response) - # Assertions + result = screenshot_url_command(mock_client, args) + assert isinstance(result, dict) assert result["error"] == "Failed to download screenshot image: HTTP 404" @@ -1046,7 +1076,10 @@ def test_reverse_padns_lookup_command_success(mock_client, mocker): # Mock response from client mock_response = { "response": { - "records": [{"answer": "192.168.1.1", "type": "A", "ttl": 3600}, {"answer": "192.168.1.2", "type": "A", "ttl": 3600}] + "records": [ + {"answer": "192.168.1.1", "type": "A", "ttl": 3600}, + {"answer": "192.168.1.2", "type": "A", "ttl": 3600}, + ] } } mock_client.reverse_padns_lookup.return_value = mock_response @@ -1117,7 +1150,10 @@ def test_forward_padns_lookup_command_success(mock_client, mocker): # Mock response from client mock_response = { "response": { - "records": [{"answer": "192.168.1.1", "type": "A", "ttl": 3600}, {"answer": "192.168.1.2", "type": "A", "ttl": 3600}] + "records": [ + {"answer": "192.168.1.1", "type": "A", "ttl": 3600}, + {"answer": "192.168.1.2", "type": "A", "ttl": 3600}, + ] } } mock_client.forward_padns_lookup.return_value = mock_response diff --git a/Packs/SilentPush/README.md b/Packs/SilentPush/README.md index 266a8baefbfd..a37efe64be29 100644 --- a/Packs/SilentPush/README.md +++ b/Packs/SilentPush/README.md @@ -1,4 +1,5 @@ The Silent Push Platform uses first-party data and a proprietary scanning engine to enrich global DNS data with risk and reputation scoring, giving security teams the ability to join the dots across the entire IPv4 and IPv6 range, and identify adversary infrastructure before an attack is launched. ## What does this pack do? + Enhances threat detection, network visibility, and incident response capabilities diff --git a/Packs/SilentPush/ReleaseNotes/1_1_0.json b/Packs/SilentPush/ReleaseNotes/1_1_0.json new file mode 100644 index 000000000000..59c63bde9316 --- /dev/null +++ b/Packs/SilentPush/ReleaseNotes/1_1_0.json @@ -0,0 +1,4 @@ +{ + "breakingChanges": true, + "breakingChangesNotes": "There is a breaking change in the silentpush-get-nameserver-reputation command, where the output context has been updated." +} \ No newline at end of file diff --git a/Packs/SilentPush/ReleaseNotes/1_1_0.md b/Packs/SilentPush/ReleaseNotes/1_1_0.md new file mode 100644 index 000000000000..d973bd8d06bb --- /dev/null +++ b/Packs/SilentPush/ReleaseNotes/1_1_0.md @@ -0,0 +1,6 @@ + +#### Integrations + +##### SilentPush + +- Updated the SilentPush integration to support V2 API for get-indicators-future-attack and fixed validation errors. \ No newline at end of file diff --git a/Packs/SilentPush/pack_metadata.json b/Packs/SilentPush/pack_metadata.json index 97ed1484426e..0ce4021aa668 100644 --- a/Packs/SilentPush/pack_metadata.json +++ b/Packs/SilentPush/pack_metadata.json @@ -2,7 +2,7 @@ "name": "Silent Push", "description": "The Silent Push platform focuses on proactive threat intelligence and threat hunting.", "support": "partner", - "currentVersion": "1.0.0", + "currentVersion": "1.1.0", "author": "Silent Push", "url": "https://www.silentpush.com/contact/", "email": "integrations@silentpush.com",