# QA and Product Audit Report

Date: July 2, 2026  
Product: VejalNai Chatbot  
Audit role: Senior QA Analyst and Product Manager

## Executive Summary

VejalNai Chatbot has grown into a broad multi-channel support console with web widget installation, inbox, bot matching, unknown-question escalation, callback handling, channel configuration, and team permissions. The current product is a strong working prototype, but it is not production-ready yet.

The largest risk is not missing UI. The largest risk is that several workflows appear complete to an admin or support agent while their real-world operational behavior is incomplete. The most important example is multi-channel human replies: agents can create human replies inside the inbox, but the product does not yet deliver those replies back to Facebook Messenger or WhatsApp customers.

## Audit Scope

Reviewed areas:

- Authentication and organization membership
- Dashboard and shared inbox
- Web chat widget and public widget API
- Widget appearance customization
- Android, iOS, Flutter, React Native, and REST integration snippets
- Facebook Messenger and WhatsApp channel configuration
- Meta webhook intake
- Teams, roles, and privileges
- Unknown-question and callback workflows
- Production readiness documentation
- Automated test coverage and route/view cache behavior

## Verification Evidence

Latest verification results during audit:

- PHPUnit passed: 44 tests, 186 assertions
- Blade template caching passed
- Route caching passed
- Main authenticated pages returned HTTP 200:
  - `/dashboard`
  - `/conversations`
  - `/teams`
  - `/settings/channels`
  - `/settings/appearance`
- Formatter gate failed before remediation planning:
  - `vendor\bin\pint --test` reported style violations in multiple files

## Severity Definitions

- P0: Blocks production release or can cause severe customer/business failure.
- P1: High-risk issue that can break important workflows or security expectations.
- P2: Important product quality gap or operational limitation.
- P3: Polish, maintainability, or future hardening.

## P0 Findings

### P0-1: Human Replies Are Not Delivered Back to Facebook or WhatsApp

Current behavior:

- Support agents can send a human reply from the inbox.
- The reply is stored locally as a `human` message.
- There is no outbound sender for Facebook Messenger or WhatsApp Cloud API.

Impact:

- Agents may believe customers received a reply when they did not.
- This breaks the core promise of a multi-channel support product.
- Customer issues can silently remain unresolved.

Recommended fix:

- Add a `ChannelMessageSender` service.
- Dispatch outbound messages based on conversation channel:
  - `web`: expose message to widget polling/history.
  - `facebook`: send to Messenger Send API using page access token.
  - `whatsapp`: send through WhatsApp Cloud API using phone number ID and access token.
- Add `delivery_status`, `provider_message_id`, and `delivery_error` metadata.
- Queue outbound delivery and retry failures.

### P0-2: Privileges Are Not Enforced Consistently

Current behavior:

- Teams can define roles and privileges.
- Some endpoints still authorize only by organization membership.

Examples:

- Conversation message sending checks membership, not `reply_to_customers`.
- Appearance settings should require `customize_widget`.
- Channel settings should require `manage_channels`.
- Unknown-question training should require `train_ai`.

Impact:

- Observer or low-privilege users may perform privileged actions.
- Product UI suggests RBAC, but backend enforcement is incomplete.

Recommended fix:

- Introduce policies or middleware for privilege checks.
- Enforce privileges on every write route.
- Add negative tests for each role.

### P0-3: Webhooks Are Not Idempotent

Current behavior:

- Facebook and WhatsApp webhook events are processed each time they are received.
- Provider message IDs are parsed but not stored with a unique constraint.
- Meta may retry webhook delivery.

Impact:

- Duplicate customer messages can be created.
- Duplicate bot replies can be sent.
- Unknown-question and callback queues can be polluted.

Recommended fix:

- Add message columns or metadata indexes for:
  - provider
  - provider_message_id
  - provider_timestamp
- Add unique index per organization/channel/provider message ID.
- Short-circuit duplicate webhook processing.
- Add tests for repeated Facebook `mid` and WhatsApp `messages[].id`.

## P1 Findings

### P1-1: Public Widget History Uses Weak Client-Side Identity

Current behavior:

- Widget history is loaded using public widget key and visitor ID.
- Visitor ID is client-generated and stored in browser session storage.

Impact:

- Widget history access is not strongly bound to a secure session.
- A guessed or leaked visitor ID could expose conversation history.

Recommended fix:

- Issue a signed conversation session token after first message.
- Use that token for history fetches.
- Rotate or expire session tokens.
- Avoid exposing all historical messages without server-side session validation.

### P1-2: Channel Setup Is Configuration-Only

Current behavior:

- Admin can store Facebook and WhatsApp IDs/tokens.
- There is no token validation, OAuth, subscription test, or health status.

Impact:

- Admin may think a channel is connected when it is not.
- Webhook configuration errors are not discoverable in the UI.

Recommended fix:

- Add connection test buttons:
  - verify token
  - validate page/phone ID
  - validate access token scopes
  - confirm webhook subscription status
- Show connected, degraded, failed, and disabled states.
- Log last inbound and last outbound event.

### P1-3: Meta Signature Verification Is Optional and Defaults to Off

Current behavior:

- Signature verification can be disabled.
- The UI offers off/sha1/sha256.

Impact:

- Production webhooks can accept spoofed requests if signature verification remains off.

Recommended fix:

- Keep signature verification optional in local development.
- Require SHA-256 verification before production-ready status.
- Add onboarding checklist warning when disabled.

### P1-4: Localized Bot Text Needs UTF-8 and Regression Validation

Current behavior:

- The `/name` and Bangla `/নাম` command area includes hard-coded localized text.
- Terminal inspection during audit showed an encoding-display risk around this area, even though the source should be treated as UTF-8.

Impact:

- Bangla user experience can break if files, terminals, or deployment environments mishandle encoding.
- Hard-coded localized strings make translation review and regression testing harder.

Recommended fix:

- Confirm all localized strings are valid UTF-8 in source, CI, and production runtime.
- Add localization tests for Bangla command flows.
- Move user-facing bot strings into translation files.

### P1-5: Formatter Gate Is Failing

Current behavior:

- PHPUnit passes, but Pint reports style issues in multiple files.

Impact:

- CI would fail if formatting is enforced.
- Inconsistent style increases maintenance cost.

Recommended fix:

- Run `vendor\bin\pint` before merge.
- Add Pint to CI.
- Treat formatting failure as release-blocking.

## P2 Product Gaps

### P2-1: AI Monitoring Is a Permission, Not a Workflow

Current behavior:

- Teams can receive `monitor_ai`.
- There is no dedicated AI monitoring queue.

Missing workflow:

- Review AI replies by confidence.
- Filter handoffs by failed intent.
- Approve or correct bot response.
- Mark false positive.
- Link correction back to intent/response training.

Recommended fix:

- Add AI Monitor page.
- Show bot message, matched intent, confidence, customer question, and response source.
- Add approve/edit/escalate controls.

### P2-2: No Audit Log

Missing audit events:

- Team role changes
- Privilege changes
- Channel token changes
- Conversation assignment
- Human takeover
- Unknown-question training
- Widget setting changes

Recommended fix:

- Add `audit_events` table.
- Record actor, organization, entity, action, before/after payload, and timestamp.
- Add audit view for owners/admins.

### P2-3: Team Member Onboarding Is Incomplete

Current behavior:

- Admin can create a user with password.

Missing production features:

- Email invitation
- Password reset
- Account activation status
- Last active tracking
- MFA or SSO readiness

Recommended fix:

- Add invitation flow.
- Add temporary invite token.
- Send email with setup link.
- Track accepted_at and last_active_at.

### P2-4: WhatsApp Business Rules Are Not Modeled

Missing:

- 24-hour customer service window
- Template message requirements outside the window
- Template management
- Delivery/read statuses
- Opt-out handling

Recommended fix:

- Add channel policy layer for WhatsApp send eligibility.
- Add templates table and approval status.
- Block/free-form sends outside allowed window.

### P2-5: Widget Customization Is Powerful but Needs Presets

Current behavior:

- Admin can tune many colors.

Risk:

- Users can create unreadable combinations.

Recommended fix:

- Add theme presets.
- Add contrast warnings.
- Add reset-to-default.
- Add preview on mobile/desktop widths.

## P3 Quality and UX Gaps

- Team privilege table can become horizontally wide on small laptops.
- No bulk actions for team members.
- No import/export for intents/responses.
- No channel-specific inbox filters.
- No SLA or priority tagging.
- No customer profile enrichment beyond basic contact fields.
- No notifications for pending human handoffs.
- No rate limiting controls visible for widget/webhook endpoints.

## Security Review Notes

Positive:

- Authenticated dashboard routes are session-protected.
- CSRF is used on dashboard forms.
- Widget API requires an enabled widget key.
- Access tokens and app secrets are encrypted at model cast level.

Risks:

- Public widget key is used for multiple public API actions.
- Webhook signature verification can be disabled.
- RBAC is not consistently enforced.
- Human replies do not have delivery/audit tracking.
- Unknown-question training writes should be restricted to `train_ai`.

## Test Coverage Assessment

Existing coverage is good for a prototype:

- Auth
- Dashboard
- Bot matching
- Widget API
- Channel webhook intake
- Teams CRUD/permissions
- Conversation foundation

Missing coverage:

- Privilege denial tests for every protected write route.
- Duplicate webhook delivery tests.
- Outbound channel delivery tests.
- Widget browser E2E tests.
- Inbox polling and optimistic-send tests.
- Appearance contrast/readability tests.
- Full multi-tenant isolation tests across all settings pages.

## Recommended Remediation Roadmap

### Sprint 1: Release Stabilization

Goals:

- Make current product safe enough for controlled pilot testing.

Tasks:

- Fix Pint formatting.
- Add privilege enforcement to all write routes.
- Add duplicate webhook protection.
- Add audit log foundation.
- Validate Bangla command encoding and add localization tests.
- Add negative authorization tests.

Exit criteria:

- PHPUnit passes.
- Pint passes.
- Route/view/config cache passes.
- No P0 issues remain.

### Sprint 2: True Multi-Channel Support

Goals:

- Make Facebook and WhatsApp operational, not just inbound-only.

Tasks:

- Build outbound send service.
- Add queue jobs and retry strategy.
- Add message delivery statuses.
- Add channel health checks.
- Add failed-delivery UI.
- Add Meta token validation.

Exit criteria:

- Agent replies reach web, Facebook, and WhatsApp users.
- Failed sends are visible and retryable.
- Duplicate webhook events do not duplicate conversations/messages.

### Sprint 3: AI Operations

Goals:

- Give humans real control over AI behavior.

Tasks:

- Add AI Monitor page.
- Show confidence, intent, matched phrase, and response template.
- Add approve/edit/escalate actions.
- Link corrections into training workflow.
- Add reporting for bot resolution and handoff causes.

Exit criteria:

- Supervisors can review and improve bot behavior without code changes.
- Unknown-question resolution improves future bot answers.

### Sprint 4: Production Onboarding and Hardening

Goals:

- Make the product easier and safer to launch.

Tasks:

- Add setup checklist.
- Require allowed origins before production.
- Require webhook signature verification.
- Add team invitations.
- Add channel setup validation.
- Add Playwright E2E smoke suite.

Exit criteria:

- A new organization can onboard widget, team, and channels without developer help.
- Production readiness status is visible in-app.

## Suggested Acceptance Criteria

Before production launch:

- Agents can reply to customers on every enabled channel.
- Every write action checks role/privilege.
- Duplicate webhooks are ignored safely.
- Unknown questions can be trained by authorized users only.
- AI monitor queue exists.
- Widget history uses a server-issued session token.
- Channel settings show verified/failed status.
- Audit log captures key admin and support actions.
- PHPUnit, Pint, route cache, config cache, and view cache all pass in CI.

## External References

- Meta Webhooks getting started: https://developers.facebook.com/docs/graph-api/webhooks/getting-started/
- Meta Webhooks for WhatsApp: https://developers.facebook.com/docs/graph-api/webhooks/getting-started/webhooks-for-whatsapp/

## Final Recommendation

Pause new feature expansion and run a stabilization sprint. The product has the right direction and a strong feature set, but it needs backend enforcement, outbound channel delivery, duplicate protection, and AI monitoring workflow before it should be positioned as production-grade.
