private void persistNewIssues(IssueStatistics statistics, List<DefaultIssue> addedIssues, IssueMapper mapper, IssueChangeMapper changeMapper) { if (addedIssues.isEmpty()) { return; } long now = system2.now(); addedIssues.forEach(i -> { int ruleId = ruleRepository.getByKey(i.ruleKey()).getId(); IssueDto dto = IssueDto.toDtoForComputationInsert(i, ruleId, now); mapper.insert(dto); statistics.inserts++; }); addedIssues.forEach(i -> issueStorage.insertChanges(changeMapper, i)); }
boolean match(DefaultIssue issue, Component file) { return matchComponent(file.getName()) && matchRule(issue.ruleKey()); }
private List<DefaultIssue> loadOpenIssues(String componentUuid, DbSession dbSession) { List<DefaultIssue> result = new ArrayList<>(); dbSession.getMapper(IssueMapper.class).scrollNonClosedByComponentUuid(componentUuid, resultContext -> { DefaultIssue issue = (resultContext.getResultObject()).toDefaultIssue(); Rule rule = ruleRepository.getByKey(issue.ruleKey()); // TODO this field should be set outside this class if ((!rule.isExternal() && !isActive(issue.ruleKey())) || rule.getStatus() == RuleStatus.REMOVED) { issue.setOnDisabledRule(true); // TODO to be improved, why setOnDisabledRule(true) is not enough ? issue.setBeingClosed(true); } // FIXME issue.setSelectedAt(System.currentTimeMillis()); result.add(issue); }); return ImmutableList.copyOf(result); }
@Override public void onIssue(Component component, DefaultIssue issue) { if (issue.isNew()) { // analyzer can provide some tags. They must be merged with rule tags Rule rule = ruleRepository.getByKey(issue.ruleKey()); if (!rule.isExternal()) { issue.setTags(union(issue.tags(), rule.getTags())); } } } }
private IssueDto doInsert(DbSession session, long now, DefaultIssue issue) { ComponentDto component = component(session, issue); ComponentDto project = project(session, issue); int ruleId = requireNonNull(getRuleId(issue), "Rule not found: " + issue.ruleKey()); IssueDto dto = IssueDto.toDtoForServerInsert(issue, component, project, ruleId, now); getDbClient().issueDao().insert(session, dto); return dto; }
private Consumer<DefaultIssue> sendNotification(BulkChangeData bulkChangeData, Map<String, UserDto> userDtoByUuid, UserDto author) { return issue -> { if (bulkChangeData.sendNotification && issue.type() != RuleType.SECURITY_HOTSPOT) { notificationService.scheduleForSending(new IssueChangeNotification() .setIssue(issue) .setAssignee(userDtoByUuid.get(issue.assignee())) .setChangeAuthor(author) .setRuleName(bulkChangeData.rulesByKey.get(issue.ruleKey()).getName()) .setProject(bulkChangeData.projectsByUuid.get(issue.projectUuid())) .setComponent(bulkChangeData.componentsByUuid.get(issue.componentUuid()))); } }; }
@Test public void issue_if_not_enough_comments() { prepareForIssue("25", FILE, 10.0, 40, 360); DefaultIssue issue = underTest.processFile(FILE, PLUGIN_KEY); assertThat(issue.ruleKey()).isEqualTo(RULE_KEY); assertThat(issue.severity()).isEqualTo(Severity.CRITICAL); // min_comments = (min_percent * ncloc) / (1 - min_percent) // -> threshold of 25% for 360 ncloc is 120 comment lines. 40 are already written. assertThat(issue.gap()).isEqualTo(120.0 - 40.0); assertThat(issue.message()).isEqualTo("80 more comment lines need to be written to reach the minimum threshold of 25.0% comment density."); }
@Test public void issue_if_not_enough_comments__test_ceil() { prepareForIssue("25", FILE, 0.0, 0, 1); DefaultIssue issue = underTest.processFile(FILE, PLUGIN_KEY); assertThat(issue.ruleKey()).isEqualTo(RULE_KEY); assertThat(issue.severity()).isEqualTo(Severity.CRITICAL); // 1 ncloc requires 1 comment line to reach 25% of comment density assertThat(issue.gap()).isEqualTo(1.0); assertThat(issue.message()).isEqualTo("1 more comment lines need to be written to reach the minimum threshold of 25.0% comment density."); }
public void add(DefaultIssue issue) { boolean isOnLeak = onLeakPredicate.test(issue); distributions.get(RULE_TYPE).increment(issue.type().name(), isOnLeak); String componentUuid = issue.componentUuid(); if (componentUuid != null) { distributions.get(COMPONENT).increment(componentUuid, isOnLeak); } RuleKey ruleKey = issue.ruleKey(); if (ruleKey != null) { distributions.get(RULE).increment(ruleKey.toString(), isOnLeak); } String assigneeUuid = issue.assignee(); if (assigneeUuid != null) { distributions.get(ASSIGNEE).increment(assigneeUuid, isOnLeak); } for (String tag : issue.tags()) { distributions.get(TAG).increment(tag, isOnLeak); } Duration effort = issue.effort(); if (effort != null) { effortStats.add(effort.toMinutes(), isOnLeak); } }
private void setType(DefaultIssue issue) { if (!issue.isFromExternalRuleEngine()) { Rule rule = ruleRepository.getByKey(issue.ruleKey()); issue.setType(rule.getType()); } issue.setIsFromHotspot(issue.type() == RuleType.SECURITY_HOTSPOT); }
@CheckForNull public Duration calculate(DefaultIssue issue) { if (issue.isFromExternalRuleEngine()) { return issue.effort(); } Rule rule = ruleRepository.getByKey(issue.ruleKey()); DebtRemediationFunction fn = rule.getRemediationFunction(); if (fn != null) { verifyEffortToFix(issue, fn); Duration debt = Duration.create(0); String gapMultiplier = fn.gapMultiplier(); if (fn.type().usesGapMultiplier() && !Strings.isNullOrEmpty(gapMultiplier)) { int effortToFixValue = MoreObjects.firstNonNull(issue.gap(), 1).intValue(); // TODO convert to Duration directly in Rule#remediationFunction -> better performance + error handling debt = durations.decode(gapMultiplier).multiply(effortToFixValue); } String baseEffort = fn.baseEffort(); if (fn.type().usesBaseEffort() && !Strings.isNullOrEmpty(baseEffort)) { // TODO convert to Duration directly in Rule#remediationFunction -> better performance + error handling debt = debt.add(durations.decode(baseEffort)); } return debt; } return null; }
@Test public void process_existing_issue() { RuleKey ruleKey = RuleTesting.XOO_X1; // Issue from db has severity major addBaseIssue(ruleKey); // Issue from report has severity blocker ScannerReport.Issue reportIssue = ScannerReport.Issue.newBuilder() .setMsg("the message") .setRuleRepository(ruleKey.repository()) .setRuleKey(ruleKey.rule()) .setSeverity(Constants.Severity.BLOCKER) .build(); reportReader.putIssues(FILE_REF, asList(reportIssue)); fileSourceRepository.addLine(FILE_REF, "line1"); underTest.visitAny(FILE); ArgumentCaptor<DefaultIssue> rawIssueCaptor = ArgumentCaptor.forClass(DefaultIssue.class); ArgumentCaptor<DefaultIssue> baseIssueCaptor = ArgumentCaptor.forClass(DefaultIssue.class); verify(issueLifecycle).mergeExistingOpenIssue(rawIssueCaptor.capture(), baseIssueCaptor.capture()); assertThat(rawIssueCaptor.getValue().severity()).isEqualTo(Severity.BLOCKER); assertThat(baseIssueCaptor.getValue().severity()).isEqualTo(Severity.MAJOR); verify(issueLifecycle).doAutomaticTransition(defaultIssueCaptor.capture()); assertThat(defaultIssueCaptor.getValue().ruleKey()).isEqualTo(ruleKey); List<DefaultIssue> issues = newArrayList(issueCache.traverse()); assertThat(issues).hasSize(1); assertThat(issues.get(0).severity()).isEqualTo(Severity.BLOCKER); }
@Test public void process_new_issue() { when(analysisMetadataHolder.isLongLivingBranch()).thenReturn(true); ScannerReport.Issue reportIssue = ScannerReport.Issue.newBuilder() .setMsg("the message") .setRuleRepository("xoo") .setRuleKey("S001") .setSeverity(Constants.Severity.BLOCKER) .build(); reportReader.putIssues(FILE_REF, asList(reportIssue)); fileSourceRepository.addLine(FILE_REF, "line1"); underTest.visitAny(FILE); verify(issueLifecycle).initNewOpenIssue(defaultIssueCaptor.capture()); DefaultIssue capturedIssue = defaultIssueCaptor.getValue(); assertThat(capturedIssue.ruleKey().rule()).isEqualTo("S001"); verify(issueStatusCopier).tryMerge(FILE, Collections.singletonList(capturedIssue)); verify(issueLifecycle).doAutomaticTransition(capturedIssue); assertThat(newArrayList(issueCache.traverse())).hasSize(1); }
private void sendIssueChangeNotification(DefaultIssue issue, Component project, Map<String, UserDto> usersDtoByUuids, NotificationStatistics notificationStatistics) { IssueChangeNotification changeNotification = new IssueChangeNotification(); changeNotification.setRuleName(rules.getByKey(issue.ruleKey()).getName()); changeNotification.setIssue(issue); changeNotification.setAssignee(usersDtoByUuids.get(issue.assignee())); changeNotification.setProject(project.getKey(), project.getName(), getBranchName(), getPullRequest()); getComponentKey(issue).ifPresent(c -> changeNotification.setComponent(c.getKey(), c.getName())); notificationStatistics.issueChangesDeliveries += service.deliver(changeNotification); notificationStatistics.issueChanges++; }
@Test public void execute_issue_visitors() { ScannerReport.Issue reportIssue = ScannerReport.Issue.newBuilder() .setMsg("the message") .setRuleRepository("xoo") .setRuleKey("S001") .setSeverity(Constants.Severity.BLOCKER) .build(); reportReader.putIssues(FILE_REF, asList(reportIssue)); fileSourceRepository.addLine(FILE_REF, "line1"); underTest.visitAny(FILE); verify(issueVisitor).beforeComponent(FILE); verify(issueVisitor).afterComponent(FILE); verify(issueVisitor).onIssue(eq(FILE), defaultIssueCaptor.capture()); assertThat(defaultIssueCaptor.getValue().ruleKey().rule()).isEqualTo("S001"); }
private boolean isInclude(DefaultIssue issue, Component file) { boolean atLeastOneRuleMatched = false; boolean atLeastOnePatternFullyMatched = false; IssuePattern matchingPattern = null; for (IssuePattern pattern : inclusionPatterns) { if (pattern.getRulePattern().match(issue.ruleKey().toString())) { atLeastOneRuleMatched = true; String filePath = file.getName(); if (filePath != null && pattern.getComponentPattern().match(filePath)) { atLeastOnePatternFullyMatched = true; matchingPattern = pattern; } } } if (atLeastOneRuleMatched) { if (atLeastOnePatternFullyMatched) { LOG.debug("Issue {} enforced by pattern {}", issue, matchingPattern); } return atLeastOnePatternFullyMatched; } else { return true; } }
@Test public void set_rule_name_as_message_when_issue_message_from_report_is_empty() { RuleKey ruleKey = RuleKey.of("java", "S001"); markRuleAsActive(ruleKey); registerRule(ruleKey, "Rule 1"); when(issueFilter.accept(any(), eq(FILE))).thenReturn(true); when(sourceLinesHash.getLineHashesMatchingDBVersion(FILE)).thenReturn(Collections.singletonList("line")); ScannerReport.Issue reportIssue = ScannerReport.Issue.newBuilder() .setRuleRepository(ruleKey.repository()) .setRuleKey(ruleKey.rule()) .setMsg("") .build(); reportReader.putIssues(FILE.getReportAttributes().getRef(), singletonList(reportIssue)); Input<DefaultIssue> input = underTest.create(FILE); Collection<DefaultIssue> issues = input.getIssues(); assertThat(issues).hasSize(1); DefaultIssue issue = Iterators.getOnlyElement(issues.iterator()); // fields set by analysis report assertThat(issue.ruleKey()).isEqualTo(ruleKey); // fields set by compute engine assertInitializedIssue(issue); assertThat(issue.message()).isEqualTo("Rule 1"); }
@Test public void load_external_issues_from_report_with_default_effort() { when(sourceLinesHash.getLineHashesMatchingDBVersion(FILE)).thenReturn(Collections.singletonList("line")); ScannerReport.ExternalIssue reportIssue = ScannerReport.ExternalIssue.newBuilder() .setTextRange(TextRange.newBuilder().setStartLine(2).build()) .setMsg("the message") .setEngineId("eslint") .setRuleId("S001") .setSeverity(Constants.Severity.BLOCKER) .setType(ScannerReport.IssueType.BUG) .build(); reportReader.putExternalIssues(FILE.getReportAttributes().getRef(), asList(reportIssue)); Input<DefaultIssue> input = underTest.create(FILE); Collection<DefaultIssue> issues = input.getIssues(); assertThat(issues).hasSize(1); DefaultIssue issue = Iterators.getOnlyElement(issues.iterator()); // fields set by analysis report assertThat(issue.ruleKey()).isEqualTo(RuleKey.of("external_eslint", "S001")); assertThat(issue.severity()).isEqualTo(Severity.BLOCKER); assertThat(issue.line()).isEqualTo(2); assertThat(issue.effort()).isEqualTo(Duration.create(0l)); assertThat(issue.message()).isEqualTo("the message"); // fields set by compute engine assertThat(issue.checksum()).isEqualTo(input.getLineHashSequence().getHashForLine(2)); assertThat(issue.tags()).isEmpty(); assertInitializedExternalIssue(issue); }
@Test public void load_external_issues_from_report() { when(sourceLinesHash.getLineHashesMatchingDBVersion(FILE)).thenReturn(Collections.singletonList("line")); ScannerReport.ExternalIssue reportIssue = ScannerReport.ExternalIssue.newBuilder() .setTextRange(TextRange.newBuilder().setStartLine(2).build()) .setMsg("the message") .setEngineId("eslint") .setRuleId("S001") .setSeverity(Constants.Severity.BLOCKER) .setEffort(20l) .setType(ScannerReport.IssueType.SECURITY_HOTSPOT) .build(); reportReader.putExternalIssues(FILE.getReportAttributes().getRef(), asList(reportIssue)); Input<DefaultIssue> input = underTest.create(FILE); Collection<DefaultIssue> issues = input.getIssues(); assertThat(issues).hasSize(1); DefaultIssue issue = Iterators.getOnlyElement(issues.iterator()); // fields set by analysis report assertThat(issue.ruleKey()).isEqualTo(RuleKey.of("external_eslint", "S001")); assertThat(issue.severity()).isEqualTo(Severity.BLOCKER); assertThat(issue.line()).isEqualTo(2); assertThat(issue.effort()).isEqualTo(Duration.create(20l)); assertThat(issue.message()).isEqualTo("the message"); assertThat(issue.type()).isEqualTo(RuleType.SECURITY_HOTSPOT); // fields set by compute engine assertThat(issue.checksum()).isEqualTo(input.getLineHashSequence().getHashForLine(2)); assertThat(issue.tags()).isEmpty(); assertInitializedExternalIssue(issue); }
private TrackedIssue toTrackedIssue(DefaultIssue issue) { TrackedIssue trackedIssue = new TrackedIssue(); trackedIssue.setKey(issue.key()); trackedIssue.setRuleKey(issue.ruleKey()); trackedIssue.setComponentKey(issue.componentKey()); trackedIssue.setSeverity(issue.severity()); trackedIssue.setResolution(issue.resolution()); return trackedIssue; } }