# bisser.io: full content # Author: Stephan Bisser # Generated: 2026-10-30 # Posts: 73 # Index: https://bisser.io/llms.txt --- title: "Your Agent Is Only as Reliable as Its Harness" url: https://bisser.io/your-agent-is-only-as-reliable-as-its-harness/ markdown: https://bisser.io/your-agent-is-only-as-reliable-as-its-harness/index.md date: 2026-10-29 lastmod: 2026-10-29 tags: AI, Agents, Agent Harness, Agent Architecture, Governance categories: AI description: "A harness is everything around the model that makes an agent reliable. What it contains, who does what, and three questions before you build." status: current reading_time_min: 7 --- # Your Agent Is Only as Reliable as Its Harness *The model gets the attention. The harness decides whether the agent does its job.* Many people do not know what an agent harness is, and you can hear it in how they describe agents. A typical sentence: "The LLM then executes the action", or "the LLM makes the tool call." That sounds harmless, but it is wrong, and it matters. The model executes nothing. It produces text that says "call this tool with these inputs". The harness takes that text, runs the tool and hands the result back to the model. So there are three things, and they get mixed up all the time. The model is the brain. The harness is everything around it. The agent is the two together. If you do not know who does what, you do not know where to look when something goes wrong. ## What a harness is The short version: the model is the brain, the harness is everything else that turns it into an agent that works reliably. A model on its own only produces text. It can suggest what to do, but it cannot open a file, send a message or remember yesterday. The harness gives it hands, rules and a memory, and keeps a hand on the wheel. ![An agent is the model plus the harness: the model on the left, the harness on the right with eight parts, four in blue that give the agent ability (instructions, tools, memory, loop) and three in red that keep it in bounds (permissions, approvals, stop rules), plus a log in gray](/images/090-harness-layers.webp) In practice, the harness consists of: - **Instructions and context:** what the agent is told and what it gets to see. - **Tools:** what it can actually do, such as searching, reading files or calling an API. - **Memory:** what it keeps from one step, or one session, to the next. - **Permissions:** which of those tools and data it may use. - **The loop:** plan, act, check the result, repeat. - **Approvals:** the points where a human has to say yes. - **A log:** a record of what happened and why. - **Stop rules:** when the agent must finish, give up or hand over. Anthropic describes the harness in more technical terms as ["the loop that calls Claude and routes Claude's tool calls to the relevant infrastructure"](https://anthropic.com/engineering/managed-agents). Microsoft puts it this way: ["If the model supplies the intelligence, the harness is everything around the model."](https://blogs.microsoft.com/blog/2026/10/28/introducing-the-microsoft-copilot-harness-unlocking-the-potential-of-truly-multi-model-agentic-work/) Different words, same idea. ## The loop at the center An agent does not answer once. It works in rounds, and the harness runs every round. The model suggests the next step. The harness decides what happens with it: it checks whether the call is allowed, asks a human where that is required, runs the tool and passes the result back. ![Who does what in one round: the model suggests a tool call, the harness runs it against the tools, and the result goes back through the harness to the model](/images/090-who-does-what.webp) Take away a piece and you see what breaks. No check, and the agent never notices its own mistakes. No approval, and it acts on things it should have asked about. No stop rule, and it loops until the budget is gone. ## Three comparisons, and where each one breaks No single comparison covers it, so here are three, each with its limit. The one I use is engine and car. **Engine and car.** The model is the engine, the harness is the rest of the car: steering, brakes, fuel line, dashboard. Nobody would call an engine on a workbench a vehicle. The comparison breaks because a car's engine rarely changes, while a model can be swapped in an afternoon. It also hides that the harness is not passive: it decides when the model is called and what it gets to see. **Horse and harness.** A horse is strong and has a will of its own. The harness does not make it stronger; it makes the strength steerable and usable. But a horse harness is passive leather, whereas an agent harness runs the loop itself, controls what the model sees and can stop it. A horse also knows the road. A model without context does not. **Operating system and application.** An operating system gives programs files, permissions, processes and a way to shut them down. A harness does the same for a model. The limit is predictability: an operating system is deterministic, a model is not. A harness cannot guarantee the outcome, it can only bound it. ## Where you can see harnesses today ### Coding agents in the terminal Coding agents show the harness most openly. In Claude Code's default manual mode, reading files is allowed, but running shell commands and editing files require approval, according to the [Claude Code permissions documentation](https://code.claude.com/docs/en/permissions). Permissions and approvals are a visible product feature here, not a hidden setting. For long tasks, the harness has to do more. In [an engineering post from 26 November 2025](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents), Anthropic describes that agents struggle to make steady progress across several context windows, because each new session starts without memory of earlier work. Their answer was a harness answer: an initializer sets up the environment, each session leaves a progress file, and descriptive git commits let later sessions recover a working state. Same model, but with a memory made of files. ### Agent platforms from vendors The big vendors now name the harness as a product layer. Microsoft is one example: its [documentation](https://learn.microsoft.com/en-us/microsoft-copilot-studio/harnesses-overview) describes it as "a runtime" between your agent design and the model that decides when to call the model, what to send it, and which tools to call. On 28 October 2026, Microsoft [introduced the same technology as the Microsoft Copilot harness](https://blogs.microsoft.com/blog/2026/10/28/introducing-the-microsoft-copilot-harness-unlocking-the-potential-of-truly-multi-model-agentic-work/) across several of its products. The direction matters more than the brand: agents that run on triggers instead of prompts and that get their own accounts. [Microsoft's October post](https://www.microsoft.com/en-us/copilot/blog/copilot-studio/delegate-shared-business-processes-to-agents-that-are-more-persistent-and-proactive-with-microsoft-copilot-studio/) says these features roll out through its Frontier program and that further access and cost controls are "planned for in the coming months". The more independently an agent works, the more the harness has to carry. ### A personal assistant with approval before every action I use a personal assistant myself, and its harness follows one rule. It may read and prepare on its own. Sending, committing to something and deleting happen only with my explicit yes. It is the same idea as the manual mode of a coding agent, just applied to everyday work: the agent gets room to prepare, and the harness holds the line where something leaves my hands. ## What makes a good harness Six things to check, in roughly this order: ### 1. Context The agent should get what it needs for this task, not everything you have. Too little context makes it guess. Too much makes it slow, expensive and inconsistent. ### 2. Tools Few tools, clearly described, each with a narrow purpose. A tool the agent can misuse will eventually be misused. ### 3. Limits Permissions follow the task, not the convenience of the builder. Read access is the default, write access is earned. ### 4. Approvals Decide up front which actions need a human yes. Irreversible and external actions, such as sending, deleting or paying, belong on that list. ### 5. Traceability You should be able to reconstruct what the agent did and why. If you cannot, you cannot improve it, audit it or trust it. ### 6. Error handling Define what happens when a tool fails, an answer is wrong or the loop goes in circles: retry, escalate, stop. An agent without stop rules is a cost risk. If I plan an agent, I build the first four first: context, tools, limits and approvals. The one teams underestimate most often is the last: error handling. ## How you recognize that the harness is the problem, not the model A simple test: keep the task, swap the model. If the same mistake survives, look at the harness. ![Decision flow: swap in a stronger model on the same task; if the mistake is gone, the model was the limit; if it stays, check the harness](/images/090-model-or-harness.webp) A few more symptoms point away from the model: - **The agent forgets.** It repeats work or contradicts itself between steps or sessions. - **It declares victory too early.** Anthropic observed exactly this in long-running coding tasks. - **It invents what it could look up.** A missing tool or missing context often looks like a hallucination. - **You cannot explain what happened.** Without a log, every failure looks like bad luck. Then keep the model and change the setup: better context, fewer tools, a clear stop rule. If the results improve, you had a harness problem. One more point. Anthropic notes that harnesses ["encode assumptions about what Claude can't do on its own"](https://anthropic.com/engineering/managed-agents). As models improve, some of those assumptions become dead weight. A harness is not something you build once; you revisit it. ## Three questions before you build 1. **What may the agent do without asking, and who is asked for everything else?** 2. **How will you find out what it did and why?** 3. **What stops it when it fails or loops?** If you cannot answer these, a better model will not save the project. Picking the model is the easy, reversible part. The harness is where your decisions live, so make them on purpose. --- title: "Building a Second Brain That Actually Works with Copilot Cowork as the Librarian" url: https://bisser.io/building-a-second-brain-that-actually-works-with-copilot-cowork-as-the-librarian/ markdown: https://bisser.io/building-a-second-brain-that-actually-works-with-copilot-cowork-as-the-librarian/index.md date: 2026-04-22 lastmod: 2026-04-22 tags: AI, Copilot, Generative AI, Gen AI, Agents, Microsoft 365 Copilot, Microsoft Foundry, Copilot Studio categories: AI, Copilot description: "How to build a personal Second Brain in OneDrive using the PARA methodology — with Copilot Cowork as the routing layer that decides where every note belongs." status: current reading_time_min: 9 --- # Building a Second Brain That Actually Works — With Copilot Cowork as the Librarian Every knowledge worker I know has the same problem: too much information, not enough structure. Meeting notes live in one tool, research in another, ideas scribbled on Post-its, articles bookmarked but never read again. The stuff you need in six months is exactly the stuff you can't find. I've tried note-taking apps, wikis, bullet journals. They all fail the same way — the *capture* is easy, the *retrieval* is the problem, and the *maintenance* is what kills it. This post is about how I solved it for myself: a personal Second Brain in OneDrive, structured around PARA, and driven by Copilot Cowork that knows where things belong. It took me a few weekends to set up. It's been paying off every day since. If you're thinking about building something similar — for yourself, not for a team — this is the blueprint. --- ## The Core Idea A Second Brain is a persistent, external memory for the things your wetware can't be trusted with: decisions, context, half-formed ideas, research, conversations. Tiago Forte popularized the term and the PARA methodology (Projects, Areas, Resources, Archive). I use that as the backbone, but I extended it in two ways: 1. **A dedicated space for AI-generated artifacts** — briefings, research outputs, meeting debriefs, weekly reviews. This stuff piles up fast once you start working with Copilot Cowork, and it needs its own home. 2. **A skill-based routing layer** — instead of me deciding where each note goes, I describe the intent in natural language ("debrief this meeting", "capture this thought", "save this article to read") and Copilot Cowork figures out the right folder. The key insight: **the hard part of a Second Brain isn't storage, it's routing.** If every new note forces you to stop and think "where does this belong?", you'll stop capturing. Automate that decision and the system runs itself. --- ## The Folder Structure Here's the tree that sits in my OneDrive: ``` /Second Brain /01_Inbox ← Unsorted captures, landing zone /02_Projects ← Time-bounded work with defined outcomes /03_Areas ← Ongoing responsibilities (leadership, team, domains) /04_Resources ← Evergreen reference material (by topic) /05_Archive ← Completed projects, retired material /06_AI ← Session logs, research, briefings, reviews, meeting debriefs /07_Templates ← Reusable note templates ``` A few opinions baked into this structure: - **`01_Inbox` is sacred.** It's the default destination whenever the routing logic isn't confident. Better to park something in the Inbox than force a bad classification. I empty it weekly. - **`03_Areas` is where most of my time goes.** Projects end; areas (people I lead, domains I own, ongoing initiatives) don't. Most knowledge workers underweight this folder. - **`06_AI` is my addition to PARA.** Once you have Copilot Cowork producing real artifacts — morning briefings, meeting debriefs, research — you need somewhere to keep them that isn't `Projects` (too narrow) or `Resources` (too generic). - **`07_Templates` reduces friction.** Every recurring note type (meeting debrief, 1-on-1, project brief, decision log, weekly review, person profile, reading list) has a template. I don't start notes from a blank page. Inside `02_Projects` and `03_Areas` I create sub-folders per project or area. Each gets an `Overview.md` at the top — a single page explaining what this area is, who's involved, what's active. Think of it as the README for that slice of your life. --- ## The AI Routing Layer Here's where it gets interesting. I use Copilot Cowork (running inside my M365 environment) with a set of custom skills — small instruction files that teach Copilot Cowork how to handle specific workflows. Each skill is just a Markdown file with a name, a description of when to trigger, and a playbook. My skills roughly map to intents: | Skill | What it does | |---|---| | `quick-capture` | Grab a thought, classify it by type, route to the right folder | | `meeting-debrief` | Pull meeting context, extract decisions and action items, save to the right project or area | | `daily-brief` | Compose a morning overview from calendar, email, and chat | | `weekly-review` | Friday ritual — empty the Inbox, reflect on progress, set next week's priorities | | `save-to-vault` | The persistence layer — copies session output to the right OneDrive folder via Graph API | The routing logic inside each skill is boring and explicit. For example, quick-capture has a table that says: *project update → `02_Projects/`*, *leadership observation → `03_Areas/People Lead`*, *readable article → append to the Reading List*, *unclear → `01_Inbox` with a tag*. No magic, just heuristics I worked out by watching what kinds of things I actually capture. Copilot Cowork doesn't write anywhere it wants. Every save goes through one skill (`save-to-vault`) that knows the folder IDs, handles the Graph API copy, and verifies the file landed. Centralizing that logic means when something breaks — a renamed folder, a new sub-structure — I fix it in one place. --- ## Templates, Not Blank Pages Every recurring note type has a template. The ones I use weekly: - **Meeting debrief** — goal, decisions, action items (with owner and deadline), observations, open questions, follow-up - **1-on-1** — topics discussed, what the person said (verbatim where possible), concerns, development areas, next time - **Project brief** — goal, stakeholders, timeline, key decisions log, current status - **Decision log** — context, options considered, decision, rationale, revisit date - **Weekly review** — what happened, what I learned, what's blocking, priorities for next week - **Person profile** — role, communication preferences, shared history, open threads - **Reading list** — queue, currently reading, finished (with takeaway), archive Templates do two things: they make capture fast (no thinking about structure), and they make retrieval useful (consistent frontmatter means I can filter by type, tag, or date later). Every template starts with YAML frontmatter: ```yaml --- type: meeting-debrief date: 2026-04-23 project: [project-slug] attendees: [names] tags: [#debrief] --- ``` Six months from now, when I'm looking for "all the decisions I made about X in Q2", the frontmatter is what makes that search tractable. --- ## A Day in the System Here's what actual usage looks like: **Morning (2 minutes)** — I ask for a daily brief. Copilot Cowork pulls today's calendar, unread emails flagged as important, and recent chat activity. It writes a short overview and archives a copy to `06_AI/Briefings`. I skim, adjust priorities, start the day. **During meetings** — I don't take notes in the moment. After each meeting I say "debrief this" and Copilot Cowork pulls the transcript (if Teams recorded it), extracts decisions and action items, classifies the meeting type (1-on-1, team, client), and saves the debrief to the right folder. Two minutes per meeting. **Between tasks** — any thought, article, observation goes through quick-capture. "Capture this: we should rework the onboarding flow, the handoff is losing people." One sentence, Copilot Cowork parses the type, routes to the right folder, confirms with a link. **Friday afternoon** — weekly review. Copilot Cowork goes through the Inbox, helps me classify each item, summarizes what happened across projects and areas, surfaces stuck items, and writes the review note to `06_AI/Weekly Reviews`. About 20 minutes, and my Inbox starts Monday at zero. The rhythm only works because the friction is close to zero. If capture took more than 15 seconds, I'd skip it. If retrieval took more than a minute, I'd stop trusting the system. --- ## What I Learned the Hard Way A few non-obvious lessons from the build: **1. Start with too-few folders, not too many.** My first version had deeply nested structure. I never used most of it. Now I default to flat — if a folder doesn't have 5+ items after a month, it probably doesn't need to exist. **2. The Inbox is a feature, not a bug.** I used to feel guilty about unclassified items. Now I treat the Inbox as a legitimate parking lot. Some things genuinely don't know what they are yet. Force-fitting them into a category makes the system worse. **3. Preserve wording, don't paraphrase.** When capturing decisions, observations, or things people said, keep the exact phrasing. "We'll revisit this in Q3" is different from "we decided to wait." Your future self needs the literal words. **4. Templates beat discipline.** I used to try to remember what sections a meeting debrief should have. I'd forget one every time. Now the template has the slots, I fill them in. Consistency is a property of the system, not of me. **5. Persist everything outside the tool.** Whatever AI assistant you use — I use Copilot Cowork — assume you'll switch tools eventually. My notes live as plain Markdown in OneDrive. No proprietary formats, no vendor lock-in. If the AI layer disappears tomorrow, the Second Brain is still there. **6. Build the routing logic gradually.** Don't try to design the perfect taxonomy up front. Start with 3–4 obvious categories, watch what you actually capture over a few weeks, add categories as patterns emerge. My current routing table took about a month to stabilize. **7. One skill per intent, not per tool.** I was tempted to build a "Teams skill", an "Outlook skill", a "OneDrive skill". Bad idea. Intents ("summarize this meeting", "capture this thought") cross tools. Build skills around what you want to do, not around where the data lives. --- ## How to Start Your Own If you want to build something similar, here's the minimal path: 1. **Pick a storage layer that isn't a note app.** OneDrive, Dropbox, Google Drive, a plain folder — anything that syncs and lets you write Markdown files. Not Notion, not Obsidian (yet), not Evernote. Start with files. 2. **Set up the PARA folders.** Projects, Areas, Resources, Archive. Add `Inbox` for captures and (optionally) an `AI` folder if you work with Copilot Cowork or a similar assistant. 3. **Create 3–4 templates.** Whatever note types you produce weekly. Meeting notes, weekly review, and a decision log is a decent starter set. 4. **Write one `Overview.md` per active project and area.** Two paragraphs each: what this is, who's involved, what's active. 5. **Pick one capture flow.** The single highest-leverage habit is capturing thoughts into the Inbox with a one-sentence intent. Everything else is polish. 6. **If you have an AI assistant with custom instructions** — like Copilot Cowork, Claude, or ChatGPT with Custom GPTs — teach it the folder structure and the routing rules. Even without automation, explicit rules written down make the system more robust. 7. **Run a weekly review.** Non-negotiable. 20 minutes every Friday: empty the Inbox, scan projects and areas, plan the coming week. Without this ritual, the system decays. That's it. Everything else is optimization. --- ## The Real Value What a Second Brain actually gives you isn't organization. It's **cognitive leverage**. You stop using your working memory for storage and start using it for thinking. You stop rediscovering the same conclusions and start building on them. You stop losing context between meetings and start entering each one already prepared. The honest test, six months in, is this: can I answer "what did I decide about X in Q1, and why?" in under a minute? If yes, the system is earning its keep. If no, something in the routing or retrieval is broken and needs fixing. I think mine passes the test. Yours will too, if you keep the system small, the capture frictionless, and the review weekly. The hardest part isn't the setup. It's trusting the system enough to stop keeping a parallel one in your head. --- *If you want to discuss your own setup or see specific examples of the templates and skill definitions I use, feel free to reach out.* --- title: "Why Most Copilot Agent Projects Fail Before They Ship" url: https://bisser.io/why-most-copilot-agent-projects-fail-before-they-ship/ markdown: https://bisser.io/why-most-copilot-agent-projects-fail-before-they-ship/index.md date: 2026-03-16 lastmod: 2026-03-16 tags: AI, Copilot, Generative AI, Gen AI, Agents, Microsoft 365 Copilot, Microsoft Foundry, Copilot Studio categories: AI, Copilot description: "The three reasons agent projects stall — and three things to do before you build." status: current reading_time_min: 5 --- # Why Most Copilot Agent Projects Fail Before They Ship *The problem is rarely the technology. It is almost always the preparation.* I have seen a lot of agent projects over the past year. Some shipped successfully, many didn't. And here is the pattern I keep noticing: the ones that fail almost never fail because of a technical limitation. They fail because of things that should have been sorted out before anyone started building. That's frustrating, because most of these failures are entirely preventable. ## The Demo Trap Let me start with the most common failure driver: **unrealistic expectations**. We have all seen the keynote demos. Someone describes what they want in natural language, an agent gets built in minutes, it connects to enterprise data, and it delivers perfect results. The audience applauds. The executives get excited. And then someone in the organization says: "Let's build that." The problem is that the demo showed the happy path of a carefully prepared scenario. It didn't show the weeks of data preparation. It didn't show the prompt engineering iterations. It didn't show the edge cases that break the agent. And it certainly didn't show the governance review that needs to happen before anything goes to production. When teams start an agent project with demo-level expectations, they are setting themselves up for disappointment. Not because the technology is bad — it is genuinely impressive — but because the gap between a demo and a production-ready agent is much larger than most people think. ## The Three Reasons Agent Projects Stall In my experience, agent projects that fail before shipping almost always hit one of these three walls: ### 1. The Platform Was Not Ready This one is painful because it is often outside your control. You start building an agent on a platform, and halfway through you realize that a critical feature is missing, still in preview, or doesn't work the way the documentation suggests. This happened a lot in the early days of Copilot Studio's agent capabilities, and it still happens today when teams try to use cutting-edge features that are not yet generally available. The platform is evolving fast, which is great — but it also means that what worked in a demo environment last month might not be production-ready this month. ### 2. The Data Was Not There This is the one that frustrates me the most, because it is the most preventable. An agent is only as good as the data it can access. If your knowledge sources are outdated, poorly structured, incomplete, or scattered across systems without proper indexing, your agent will deliver poor results — no matter how good your instructions are. I have seen teams spend weeks crafting perfect agent instructions, only to realize that the SharePoint library the agent is grounded on hasn't been maintained in two years. The agent works technically. It just returns useless information. **Data quality is not an agent problem. It is a prerequisite.** And it needs to be reviewed before you start building, not after. ### 3. The Wrong Platform Was Chosen The Microsoft ecosystem offers multiple ways to build agents — Agent Builder, Copilot Studio, Microsoft Foundry, the Agents SDK, and more. Each targets a different audience and a different level of complexity. Choosing the wrong platform for your use case is a recipe for frustration. I have seen citizen developers struggle in Copilot Studio with scenarios that really needed Microsoft Foundry. And I have seen pro developers over-engineer solutions in code when a declarative agent in Copilot Studio would have been done in an afternoon. The platform decision should be one of the first things you make — and it should be based on the complexity of your use case, not on which tool your team happens to be familiar with. ## Three Things to Do Before You Build Anything If I could give every team starting an agent project just three pieces of advice, it would be these: **Define a clear goal and concept first.** What exactly should this agent do? What is the specific use case? Who is the user? What does success look like? If you can't answer these questions in one paragraph, you are not ready to build. Too many projects start with "let's build an agent for HR" instead of "let's build an agent that helps new hires find answers about onboarding policies in their first week." Specificity matters. **Choose the right platform deliberately.** Don't default to the tool you know. Evaluate your use case against the capabilities of each platform. Does it need real-time API integrations? Does it need complex orchestration? Can it be solved with a simple declarative agent? Match the platform to the problem, not the other way around. **Review your data quality before you start.** Go look at the actual data sources your agent will use. Are they current? Are they well-structured? Are there duplicates or contradictions? Is access properly configured? This review takes a day or two at most, but it can save you weeks of debugging an agent that "doesn't work" when the real problem is the data underneath. ## When Failure Is Actually Fine I want to be clear: not all failure is bad. If you are experimenting with cutting-edge technology — trying something that hasn't been done before, pushing the boundaries of what the platform can do — then failure is part of the process. That is how we learn, and that is how the ecosystem moves forward. The failure that bothers me is the preventable kind. Teams that skip the basics, get burned, and then conclude that "agents don't work" — when the agents were never given a fair chance in the first place. ## It Will Get Better The good news is that the failure rate for agent projects will come down over time. The platforms are maturing rapidly. The community is building up knowledge. Organizations are learning from their early experiments. Every failed project teaches someone what not to do next time. But we can accelerate this by being honest about why projects fail. It is not because the technology is not ready. In most cases, it is because we were not ready — we didn't define the goal clearly enough, we picked the wrong platform, or we didn't check the data. Those are human problems, not technology problems. And human problems have human solutions. **What was the biggest lesson you learned from an agent project that didn't go as planned?** --- title: "Agent Governance Is the Next Big Bottleneck" url: https://bisser.io/agent-governance-is-the-next-big-bottleneck/ markdown: https://bisser.io/agent-governance-is-the-next-big-bottleneck/index.md date: 2026-03-13 lastmod: 2026-03-13 tags: AI, Copilot, Generative AI, Gen AI, Agents, Microsoft 365 Copilot, Governance, Agent Governance, Agents 365 categories: AI, Copilot description: "Six governance pillars, shadow agents, and why you need a dedicated AI lead." status: current reading_time_min: 5 --- # Agent Governance Is the Next Big Bottleneck *Everyone is building agents. But who decides what they are allowed to do?* Here's an irony I keep seeing: organizations that don't have agent governance in place don't end up with chaos. They end up with nothing. Because when there are no clear rules about what people can and cannot do with agents, IT does the only rational thing — they block everything. And I get it. If you're responsible for security and compliance, and suddenly business users start building agents that connect to your ERP, your CRM, and your internal databases without any oversight, your instinct is to shut it down. The problem is that blocking everything is just as damaging as allowing everything. You're just trading one risk for another. ## The Real Problem Is Not Tooling When people talk about agent governance, the conversation usually jumps to platform features — admin centers, policies, DLP rules. And yes, those matter. But in my experience, the real bottleneck is much more fundamental: **most IT departments don't have the knowledge to think about agent governance holistically**, and most organizations don't have a dedicated person who owns this topic. Think about it. Agent governance sits at the intersection of IT security, data governance, application lifecycle management, and business process design. That's a lot of domains to cover. And right now, in most organizations, nobody owns this intersection. Security owns their piece. IT ops owns their piece. The business owns their piece. But nobody is looking at the full picture. That's how you end up with either total lockdown or total chaos. There's no middle ground without someone actively designing it. ## The Six Pillars of Agent Governance If I had to help an organization build an agent governance framework from scratch, these are the six areas I'd focus on: | Pillar | Key Question | | --- | --- | | **Roles & Permissions** | Who is allowed to build, test, and deploy agents? What can citizen developers do vs. pro developers? | | **Scope & Data Access** | Which data sources can an agent access? Which systems can it interact with? Which actions can it perform? | | **Lifecycle Management** | How does an agent go from idea to production? What are the gates for testing, review, and approval? | | **Incident Management** | What happens when an agent behaves unexpectedly? Who is responsible? Is there an audit trail? | | **Quality Standards** | What are the minimum requirements for agent instructions, grounding, and testing before deployment? | | **Agent Inventory** | Do you have a catalog of all active agents? Can you spot duplicates and shadow agents? | None of these are revolutionary ideas individually. But most organizations I talk to haven't thought through even half of them. And without all six in place, you're flying blind. ## Shadow Agents Are the New Shadow IT Remember when business users started spinning up their own cloud services without IT knowing about it? We called it Shadow IT, and it took years to get under control. The same thing is happening with agents right now — I'd call it **Shadow Agents**. Without governance, two things happen consistently: **Teams build duplicate agents.** The marketing team builds an agent for content creation. The communications team builds a different agent for the same thing. Nobody knows the other exists. You end up with redundant work, inconsistent outputs, and wasted effort. **Agents access unauthorized systems.** Someone builds an agent that connects to a data source or system that IT hasn't approved. Not maliciously — they just didn't know they needed to ask. But now you have an agent pulling data from a system without proper access controls, and nobody in security is aware. This is exactly the pattern we saw with Shadow IT, just on a new level. And the solution is the same: you don't fix it by blocking everything. You fix it by creating a clear framework that makes it easy to do the right thing. ## You Need a Dedicated AI Lead The single most impactful thing an organization can do for agent governance is to create a dedicated role — someone who owns agent governance end to end. And this person needs to be a **bridge between IT and business**. Not a pure IT role. Not a pure business role. Someone who understands the technical possibilities of agent platforms, but also deeply understands the business processes and use cases. Someone who can translate between the security team saying "we need to control data access" and the business team saying "we need agents that actually do useful work." Where this person sits in the org chart matters less than the mandate they have. They need the authority to define policies, the technical depth to evaluate agent implementations, and the business acumen to prioritize the right use cases. It's a rare combination, but it's the role that makes everything else possible. ## Microsoft Is Building the Control Plane To be fair, Microsoft isn't ignoring this. With Agents 365, they're building a control plane for agents where governance topics are increasingly being addressed. Centralized management, visibility into what agents exist and what they do, policy enforcement — these capabilities are coming. But here's what I keep saying: **governance is primarily an organizational challenge, not a technology challenge.** The best admin center in the world doesn't help if nobody has defined the policies it should enforce. Microsoft can build the tools, but organizations need to do the thinking. The platforms will keep getting better. Agents 365 will mature. Copilot Studio will add more governance features. But the organizational work — defining roles, establishing processes, building the knowledge in your IT team — that's on you. And the sooner you start, the less painful it will be. ## The Bottom Line Agent governance is going to be the defining factor that separates organizations that successfully adopt agents from those that don't. Not because governance is exciting — it's not. But because without it, you either get paralysis (IT blocks everything) or chaos (shadow agents everywhere). Neither gets you to the agentic organization. The good news: you don't need to have everything figured out on day one. Start with the basics — a role model, an approval process, an agent inventory. Then iterate. But start now, because every week without governance is a week where the gap between what's being built and what's being managed grows wider. **Does your organization have a dedicated person or team responsible for agent governance — or is it still everyone's and nobody's job?** --- title: "Testing AI Agents Is a Problem Nobody Wants to Talk About" url: https://bisser.io/testing-ai-agents-is-a-problem-nobody-wants-to-talk-about/ markdown: https://bisser.io/testing-ai-agents-is-a-problem-nobody-wants-to-talk-about/index.md date: 2026-03-11 lastmod: 2026-03-11 tags: AI, Copilot, Generative AI, Gen AI, Agents, Microsoft 365 Copilot, Testing, Agent Testing, ALM categories: AI, Copilot description: "Why agent testing is neglected, why classical methods fail, and what to do instead." status: current reading_time_min: 6 --- # Testing AI Agents Is a Problem Nobody Wants to Talk About *Everyone is building agents. Almost nobody is testing them properly.* Here's something that's been bugging me for a while: in classical software development, testing is non-negotiable. Nobody would ship a production application without at least some level of automated testing, code reviews, and quality gates. It's fundamental. And yet, when it comes to AI agents, testing is often completely neglected. It's like we collectively decided that the rules don't apply anymore. I think it's time we talk about this. ## Why Agent Testing Falls Through the Cracks The main reason is actually straightforward: the people building agents today aren't necessarily developers. Microsoft has done an incredible job democratizing agent development — from Agent Builder to Copilot Studio, anyone can build an agent without writing a single line of code. And that's genuinely great for innovation. But here's the flip side: many of these builders have never been exposed to the discipline of software testing. They don't know what a test plan looks like. They've never written a test case. Not because they're not smart — they absolutely are — but because testing was never part of their world. When a business analyst builds an agent in Copilot Studio, their definition of "done" is usually "it works when I try it." And that's not the same as "it's been properly tested." ## Deterministic Testing Doesn't Work Here Even if you do come from a development background, you'll quickly realize that classical testing approaches don't translate well to AI agents. In traditional software, you test deterministically: given input X, you expect output Y. If the output matches, the test passes. Simple. With AI agents, that model breaks down completely. Ask the same agent the same question twice, and you might get two different answers — both of which could be perfectly correct. The underlying language model is non-deterministic by nature. So if you try to apply unit test thinking to agent testing, you'll either go crazy or give up. Neither is helpful. What we need is a fundamentally new testing mindset. ## New Categories for a New Paradigm Instead of testing for exact outputs, I think we need to focus on qualitative evaluation categories that actually matter for agents: | Category | What It Tests | | --- | --- | | **Groundedness** | Are the agent's answers based on the provided knowledge sources, or is it hallucinating? | | **Tool Usage** | Does the agent call the right tools and APIs for a given request? | | **Semantic Similarity** | Is the answer semantically correct, even if the wording differs from the expected response? | | **Relevance** | Does the agent actually answer the question that was asked? | | **Coherence** | Is the response logically structured and consistent? | | **Safety** | Does the agent resist adversarial prompts and stay within its defined boundaries? | These categories shift the focus from "is the output identical?" to "is the output good?" — and that's exactly the shift we need. ## What State-of-the-Art Looks Like Today The good news is that tooling is catching up. Here's what I see as the current state-of-the-art for agent testing: **Evaluation frameworks** like Azure AI Evaluation SDK or DeepEval can automatically score agent responses on metrics like groundedness, relevance, and coherence. Essentially, you use one LLM to evaluate the output of another. It's not perfect, but it scales. **Golden datasets** — curated sets of question-answer pairs that serve as benchmarks. The key difference to classical test data: you don't check for exact matches but for semantic similarity. The agent doesn't need to produce the same words, just the same meaning. **Tool call assertions** — verifying that the agent invokes the correct tools for a given input, regardless of the textual response. If someone asks "What's my leave balance?", the agent should call the HR API, not the finance API. This is actually quite testable. **Red teaming** — deliberately trying to break the agent. Can you make it hallucinate? Can you trick it into revealing information it shouldn't? Can you push it off-topic? This adversarial approach catches issues that happy-path testing never will. **Human-in-the-loop evaluation** — running example conversations and having humans manually assess the quality. This is labor-intensive but catches nuances that automated metrics miss. ## The Platform Gap Here's something worth noting: not all Microsoft platforms are equal when it comes to testing support. Microsoft Foundry already offers a solid evaluation toolkit — you can run evaluations against datasets, measure quality metrics, and integrate this into your development workflow. Copilot Studio and Agent Builder? Not so much. If you're building agents in these platforms, you're largely on your own when it comes to structured testing. I hope this gap closes soon, because these are exactly the platforms where citizen developers build agents — the same people who need testing guidance the most. ## Make Testing Non-Optional If there's one practical takeaway from this post, it's this: **don't make agent testing a recommendation — make it a requirement.** Organizations need to establish an ALM (Application Lifecycle Management) process for agents that explicitly includes testing as a mandatory step. Not a "nice to have." Not a "we'll add testing later." A hard gate that every agent must pass before it reaches production. This means: - **Define minimum testing criteria** for every agent before it gets deployed - **Provide testing templates** — example conversations, evaluation rubrics, tool call checklists — so that citizen developers have a starting point - **Build testing into the approval workflow** — no test results, no deployment Yes, this adds friction. But it's the same kind of friction that prevents broken software from reaching production in every other part of your technology stack. ## The Bottom Line Agent testing is where software testing was 20 years ago — everyone knows they should do it, but many organizations are still figuring out how. The difference is that we're building agents at a pace that's far faster than we ever built traditional applications, which means the gap between what's being shipped and what's been properly tested is growing quickly. I believe agent testing will eventually become as natural as unit testing is today. The tools will improve, the platforms will integrate better evaluation capabilities, and organizations will develop testing muscle through experience. But we can't just wait for that to happen. We need to start now — with the tools we have, with the frameworks that exist, and with the mindset that agents deserve the same quality standards as any other piece of software we put in front of our users. **How does your organization handle agent testing today — and do you have a process in place, or is it still the wild west?** --- title: "The Agentic Organization — When Agents Become Part of the Team" url: https://bisser.io/the-agentic-organization-when-agents-become-part-of-the-team/ markdown: https://bisser.io/the-agentic-organization-when-agents-become-part-of-the-team/index.md date: 2026-03-09 lastmod: 2026-03-09 tags: AI, Copilot, Generative AI, Gen AI, Agents, Microsoft 365 Copilot, Agentic Organization categories: AI, Copilot description: "A 5-stage maturity model for the journey from Copilot rollout to agents as teammates." status: current reading_time_min: 5 --- # The Agentic Organization — When Agents Become Part of the Team *We've been talking about agents. It's time to talk about what happens when they join the org chart.* The word "agentic" is everywhere right now. Every platform, every keynote, every product update — everything is suddenly "agentic." But here's what I've noticed: everyone uses the term, and everyone means something slightly different by it. So before we talk about the agentic organization, I think we need to take a step back and create some clarity. ## Where We Are Today Right now, most interactions with AI agents follow a simple pattern: one person, one agent, one task. You ask your Copilot agent a question, it gives you an answer. You trigger a workflow, the agent executes it. It's a 1:1 relationship — useful, productive, but fundamentally still a tool interaction. And that's fine. That's the current state for most organizations, and there's a lot of value in getting this right. But I don't think this is the end state. I think the next logical step on our agentic AI journey is something bigger: organizations where people and agents work together in teams. Not as tool and user, but as teammates with different roles and responsibilities. That's what I mean when I talk about the agentic organization. ## The Agentic Maturity Model To make this more tangible, here's how I see the journey from "we just got Copilot" to a fully agentic organization. Most companies will recognize themselves somewhere on this scale: | Stage | Description | | --- | --- | | **Stage 1: Copilot Rollout** | You're deploying Microsoft 365 Copilot, figuring out adoption, licensing, and governance basics | | **Stage 2: Agent Ideation** | You're thinking about agent use cases, exploring Copilot extensibility, and piloting your first agents | | **Stage 3: Targeted Deployment** | You're running agents in selected teams or specific process steps — real work, limited scope | | **Stage 4: Governance & Process Readiness** | You have agent governance under control and you're actively redesigning processes to be agent-ready | | **Stage 5: Agentic Organization** | People and agents collaborate in interdisciplinary teams, working together on value-creating processes | Most organizations I talk to are somewhere between Stage 1 and Stage 2. A few ambitious ones are approaching Stage 3. And that's perfectly fine — but it helps to know where you're heading. ## What Stage 5 Actually Looks Like Let me make this concrete, because "people and agents working together" can sound abstract. **Finance & Controlling.** Imagine agents handling the data analysis — pulling numbers, identifying anomalies, running comparisons across periods. The humans on the team don't spend their time building spreadsheets anymore. Instead, they review the agent's findings, add context that only they have ("this spike is because we acquired a company last quarter"), and make the decisions. The agent does the heavy lifting. The human does the thinking. **Marketing.** Agents manage the channel execution — scheduling posts, adapting content for different platforms, monitoring engagement metrics. They report back to the team with performance data and recommendations. The humans focus on strategy, creative direction, and the decisions that require judgment and brand understanding. **Value-creating processes in general.** The pattern is the same across domains: agents execute, humans supervise and decide. It's a fundamental shift from "the human does the work and the agent helps" to "the agent does the work and the human steers." That's the real transformation. Not just using agents as better tools, but integrating them as team members with defined responsibilities. ## The Trust Question I know what some of you are thinking: "Can we really trust agents to do the work while we just supervise?" And yes, there are enough skeptics out there — and healthy skepticism isn't a bad thing. But I think we should approach this positively rather than letting fear block progress. The key is finding the right balance. You don't hand over your entire financial reporting to an agent on day one. You start small, build trust through experience, and gradually expand the agent's responsibilities as you learn what works and what doesn't. This is no different from onboarding a new team member, honestly. You wouldn't give a new hire full autonomy on their first day either. You'd start with defined tasks, review their output, and expand their scope over time. The same principle applies to agents. ## Start Now — Even on Stage 1 Here's the thing: you don't need to be on Stage 4 to start thinking about Stage 5. In fact, the organizations that will get to the agentic organization fastest are the ones that start building the foundation now. What does that mean practically? **Foster curiosity.** Give your people the space and permission to experiment with agents today. Not in a formal "innovation lab" that nobody takes seriously, but in their actual daily work. Let them try things, let them fail, let them learn. That's how organizational capability is built. **Think about your processes.** Look at your workflows and ask: which of these could be restructured so that agents handle the execution while humans handle the decisions? You don't need to implement this tomorrow, but starting to think this way changes how you approach every process improvement from now on. **Don't wait for perfection.** The tools will keep evolving. The governance frameworks will mature. But the organizational learning — understanding how your teams can work with agents, what works in your specific context, what doesn't — that only comes from doing. ## The Bottom Line The agentic organization isn't science fiction. It's the logical next step in a journey that many organizations are already on. We went from "AI as a feature" to "Copilot as an assistant" to "agents as tools." The next step is "agents as teammates." I believe this will affect every organization to some degree. The question isn't whether the agentic organization is coming — it's whether your organization will be ready when it arrives. **Where is your organization on the agentic maturity scale today, and what's your next step to move forward?** --- title: "The Real ROI of Microsoft 365 Copilot Extensibility" url: https://bisser.io/the-real-roi-of-microsoft-365-copilot-extensibility/ markdown: https://bisser.io/the-real-roi-of-microsoft-365-copilot-extensibility/index.md date: 2026-03-02 lastmod: 2026-03-02 tags: AI, Copilot, Generative AI, Gen AI, Agents, Microsoft 365 Copilot categories: AI, Copilot description: "When does extending Microsoft 365 Copilot pay off? My take on real ROI, readiness, and what to measure." status: current reading_time_min: 4 --- # The Real ROI of Microsoft 365 Copilot Extensibility *When does extending Microsoft 365 Copilot actually pay off? Here are my thoughts.* I've been consuming a lot of content around Microsoft 365 Copilot extensibility lately. And while the community is doing a great job producing tutorials, samples, and getting-started guides, I noticed something: there's a ton of Hello World-style content out there, but very little about the actual return on investment. So I wanted to share my personal take on when extensibility is worth it — and what the real value looks like. ## Are You Even Ready? Here's the thing: many organizations I talk to are still in the middle of rolling out Microsoft 365 Copilot to their users. They're figuring out adoption, licensing, governance — the basics. And that's totally fine. But for these organizations, extensibility shouldn't be the priority yet (even though the conference demos make it look like everyone should be building agents right now). To me, the "extensibility moment" comes when you've reached a certain maturity. You've got a Copilot rollout going. Your users are actively working with it. You've identified use cases that work out of the box. And then — this is key — you start noticing the gaps. The moments where users need data or processes from third-party systems, LOB apps, or your ERP that Microsoft 365 Copilot simply can't reach by default. That's when extensibility becomes relevant. Not because a demo looked cool, but because your users actually need it. ## It's About Personalization, Not Just Time Savings When people talk about ROI for extensibility, the conversation usually goes straight to time savings. "The agent saves 15 minutes per day." And sure, that's measurable. But to me, that misses the bigger picture. The real value of Microsoft 365 Copilot extensibility is **personalization**. Out of the box, Microsoft 365 Copilot is a powerful but generic tool. It works the same for every organization. But it doesn't know your ERP system. It doesn't understand your specific business processes. It can't interact with the industry-specific applications your people use every day. Extensibility changes that. It transforms Microsoft 365 Copilot from a general-purpose assistant into something that understands and operates within *your* specific context. And that's where the real productivity and efficiency boost comes from. ## From Information to Delegation Let me make this concrete (because I think this example explains it best): **Without extensibility**, a user asks Microsoft 365 Copilot: *"How do I submit a purchase order in our ERP system?"* — and Copilot tells them which screens to navigate and which fields to fill in. Useful? Yes. Better than a manual? Definitely. **With extensibility**, the same user asks: *"Submit a purchase order for 500 units of component X from supplier Y."* — and an agent connected to the ERP actually does it. Validates the request, triggers the process, confirms completion. The user didn't get information about how to do their job. The agent did the job on their behalf. That shift — from information to action, from assistance to delegation — is where the ROI becomes obvious. Not in minutes saved, but in entire process steps eliminated. ## Start Small, Please One thing I've seen way too often: organizations approaching extensibility with too much ambition too early. They want an agent covering 20 processes across five systems on day one. And these projects tend to stall. What works better (and I can't stress this enough): start with an MVP. Pick one process. One integration. One use case where the current workflow is painful. Build something simple. Ship it. Learn. Iterate. Then expand. ## Measuring What Actually Matters If you're going to invest in extensibility, you should know whether it's working. And "users like it" is not a metric. Here's what I think you should be looking at: | Metric | What it tells you | | --- | --- | | **System switches per task** | How many apps does a user still need to open? Every eliminated switch = less friction and fewer errors | | **Usage frequency** | Not just "who tried it" but "who keeps coming back" — recurring usage signals real value | | **Error rate in processes** | Automated handoffs should reduce manual mistakes — track before and after | | **Time to competency** | How fast can new employees become productive with agent support vs. without? | Especially that last one is often overlooked. When a new hire needs to learn complex ERP processes, an agent that can guide or even execute these processes dramatically shortens the learning curve. That's a powerful ROI argument. ## So, What's Your Extensibility Moment? I'm not saying every organization needs Microsoft 365 Copilot extensibility right now. Many don't — yet. But I do think everyone should be actively thinking about it. Look at your users' daily workflows. Where do they leave Microsoft 365 to get things done? Where are processes slow or error-prone because the tools aren't connected? Those are your extensibility opportunities. **Which processes in your organization do you think could get a real boost from Microsoft 365 Copilot extensibility?** --- title: "The Citizen Developer Promise for Copilot Agents is an Illusion" url: https://bisser.io/the-citizen-developer-promise-for-copilot-agents-is-an-illusion/ markdown: https://bisser.io/the-citizen-developer-promise-for-copilot-agents-is-an-illusion/index.md date: 2026-02-28 lastmod: 2026-02-28 tags: AI, Copilot, Generative AI, Gen AI, Agents, Microsoft 365 Copilot categories: AI, Copilot description: "Microsoft's promise that business users can independently build production-ready Copilot agents through Agent Builder and Copilot Studio breaks down the moment you need real integrations, complex debugging, or enterprise governance. Instead of selling the illusion of fully self-service agent deve..." status: current reading_time_min: 6 --- # The Citizen Developer Promise for Copilot Agents is an Illusion *Low-code doesn't mean low-complexity. Here's where the line really is.* Microsoft has been telling a beautiful story: business users can describe what they want in natural language, and Agent Builder in Microsoft 365 Copilot will create an agent for them. When they outgrow the basics, they can seamlessly move to Copilot Studio with one click. No code required. Anyone can build an agent. I've been watching this narrative evolve for over a year now, and I think it's time for a reality check. The citizen developer promise for Copilot agents, as it's currently marketed, is an illusion. Not a lie — an illusion. There's an important difference. ## The Happy Path Works Beautifully Let me be fair first. If you want to build a declarative agent that answers questions based on a curated set of SharePoint documents, Agent Builder does a genuinely impressive job. You describe the agent's purpose, point it at your knowledge sources, maybe adjust the instructions, and you have something functional in minutes. For this specific use case — essentially a scoped Q&A bot over your own content — the citizen developer story absolutely holds up. The problem starts the moment you need anything beyond that happy path. ## Where Things Fall Apart **The moment you need real integrations.** Your HR manager wants an agent that doesn't just answer questions about company policies but actually checks leave balances in Workday and submits time-off requests. That requires MCP server connections, authentication configuration, understanding OAuth flows, and dealing with connector infrastructure. I've watched technically savvy business analysts — people who are genuinely good with Power Automate and Power Apps — hit a wall when they try to configure MCP authentication with dynamic client registration. These aren't concepts that a weekend workshop prepares you for. **The moment you need nuanced instructions.** Agent instructions look simple — you're just writing natural language, right? But crafting instructions that reliably produce the right behavior across edge cases is essentially prompt engineering. It requires understanding how the underlying model interprets ambiguity, how grounding works, what happens when the agent can't find an answer, and how to handle multi-step reasoning. This is a skill that takes practice to develop, and most business users don't have the feedback loops to develop it. **The moment you need to debug.** Something isn't working. The agent is returning wrong information, or it's not calling the right MCP tool, or the authentication keeps failing. Where do you even start? Copilot Studio's tracing and analytics have improved significantly, but interpreting activity maps, understanding tool invocation patterns, and diagnosing orchestration failures requires a developer mindset. There's no low-code way to debug a complex agent. **The moment you need governance.** Your citizen-developed agent works great in testing. Now it needs to go through the approval workflow in the admin center. IT needs to review the data sources, security needs to assess the permissions model, and compliance needs to verify the agent doesn't expose sensitive data. The citizen developer built the agent — but they can't shepherd it through enterprise governance alone. ## The "Copy to Copilot Studio" Gap Microsoft's answer to the complexity cliff is the "Copy to Copilot Studio" feature. Start simple in Agent Builder, then graduate to the full platform when you need more power. It sounds seamless, but it creates a handoff problem that nobody talks about. The business user who built the prototype in Agent Builder understands the business logic. The developer who takes over in Copilot Studio understands the technology. Neither fully understands the other's domain. The result is usually one of two things: the developer rebuilds from scratch (wasting the prototype work), or the developer tries to extend the existing agent without fully understanding the business intent (leading to subtle bugs that only surface in production). This isn't a new problem — it's the same gap that has plagued every low-code-to-pro-code transition since the concept was invented. But it's being presented as solved, and it isn't. ## What Would Actually Work I'm not arguing against democratizing agent development. I think it's the right direction. But I think we need a more honest model of what citizen development looks like for agents: **Tier 1: Genuine citizen development.** Q&A agents over curated content. Simple declarative agents with file-based knowledge. This is where business users can truly self-serve, and we should encourage it. But let's be clear that this is the scope. **Tier 2: Guided development.** Agents that need integrations, complex logic, or multi-step workflows. This requires a "buddy system" — a business user who understands the process working alongside a developer or a power user who understands the platform. Neither can do it alone effectively. **Tier 3: Professional development.** Agents with custom MCP servers, complex authentication, multi-agent orchestration, or enterprise-scale governance requirements. This is developer territory, full stop. Pretending otherwise sets everyone up for frustration. **Invest in the middle tier.** Most organizations focus on either the self-service story (Tier 1) or the pro-dev story (Tier 3) and completely ignore Tier 2. But Tier 2 is where the most valuable agents live — the ones that are specific enough to a business process to be truly useful, but complex enough to need some technical guidance. **Create feedback loops.** The biggest gap in the citizen developer experience isn't tooling — it's learning. When a business user builds an agent that doesn't work well, they rarely know *why* it doesn't work well. Build internal communities of practice where people share what worked, what didn't, and why. This accelerates learning far more than any tutorial. **Redefine success.** If your measure of citizen developer success is "business users building production agents independently," you'll be disappointed. If your measure is "business users prototyping agent ideas that can be quickly validated and refined with technical support," you'll be much happier — and you'll ship more useful agents. ## The Bottom Line The tools Microsoft is building are genuinely impressive. Agent Builder, Copilot Studio, the MCP ecosystem — these are powerful platforms that lower the barrier to agent development significantly. But "lower the barrier" is not the same as "eliminate the barrier," and the marketing often conflates the two. The organizations that get the most value from Copilot agents won't be the ones where everyone builds their own agents. They'll be the ones that create the right collaboration model between business expertise and technical capability, with clear expectations about what each tier of development can realistically deliver. Let's stop selling the illusion and start building the support structures that make citizen-involved agent development actually work. --- *Have you tried building Copilot agents as a non-developer? Or are you a developer supporting citizen developers? I'd love to hear about your experience.* --- title: "Microsoft 365 Copilot Extensibility - Possibilities and Pitfalls" url: https://bisser.io/microsoft-365-copilot-extensibility-possibilities-and-pitfalls/ markdown: https://bisser.io/microsoft-365-copilot-extensibility-possibilities-and-pitfalls/index.md date: 2025-12-02 lastmod: 2025-12-02 tags: AI, Copilot, Generative AI, Gen AI, Agents, Microsoft 365 Copilot categories: AI, Copilot description: "Microsoft 365 Copilot can be extended through three pillars—Connectors (for bringing external data in), Agents (for performing actions via OpenAPI, MCP, or Copilot Studio), and APIs (for embedding Copilot capabilities in your apps)—but the key gotcha is that Actions only work inside Agents. Start..." status: current reading_time_min: 5 --- # Microsoft 365 Copilot Extensibility: Navigating the Possibilities and Pitfalls As organizations rush to adopt Microsoft 365 Copilot, a critical question emerges: **How do we extend Copilot to work with our unique business data and processes?** The answer lies in understanding the three extensibility pillars—Connectors, Agents, and APIs—and knowing when (and when not) to use each. And that's why I created an infographic for that: {{< figure src="/images/m365copilotextensibility.png" >}} ## Understanding the Anatomy of Microsoft 365 Copilot Before diving into extensibility, let's understand what we're extending. At its core, Microsoft 365 Copilot consists of: ### The Core Engine - **Orchestrator**: The traffic controller that manages data governance, safety, and responsible AI (RAI) policies - **Foundation Models**: The AI backbone powered by GPT-4o, GPT-4.1, and newer models like o1 and o3-mini ### Where Extensibility Plugs In Copilot's architecture exposes three key areas for extension: | Area | Built-in | Extensible Via | |------|----------|----------------| | **Knowledge** | Memory | Grounding ← Connectors | | **Skills** | Workflows, M365 Apps | Triggers & Actions ↔ Agents | | **APIs** | — | Your Apps → Copilot APIs | ## The Three Extensibility Pillars ### 1. Connectors: Bringing External Data to Copilot **Direction: External Data → Copilot** Graph Connectors allow you to ingest and index external data into Microsoft Graph, making it available for Copilot to reason over. **What you get:** - 100+ prebuilt connectors for popular systems - Custom connector development via Graph API - Support for CRM, ERP, databases, and file systems **Best for:** - Making enterprise data searchable and available to Copilot - Connecting line-of-business applications - Enabling Copilot to answer questions about your proprietary data **⚠️ Pitfall:** Connectors only provide *read* access. If you need Copilot to *take actions* in external systems, you'll need Agents. --- ### 2. Agents: AI Assistants That Do Work **Direction: AI Assistants ↔ M365 (Bidirectional)** Agents are specialized AI assistants that can automate workflows and perform tasks. They're the most powerful—and most complex—extensibility option. #### Inside Agents: Core Components Every agent consists of three parts: 1. **Knowledge**: Data sources like SharePoint, OneDrive, Graph Connectors, and web content 2. **Instructions**: Custom persona, tone, scope limits, and behavioral guardrails 3. **Actions**: Real-time API calls that let the agent *do* things > ⚠️ **Critical Insight:** Actions ONLY work inside Agents! You cannot add actions to base Copilot—they must be wrapped in an agent. #### Two Types of Agents | Type | AI Engine | Best For | |------|-----------|----------| | **Declarative Agent** | Uses Copilot's AI | Quick deployment, Microsoft-managed AI | | **Custom Engine Agent** | Your own AI | Full control, custom models, complex scenarios | #### Three Ways to Build Actions | Approach | Technology | Characteristics | |----------|------------|-----------------| | **OpenAPI Action** | REST API + OpenAPI Spec | Existing APIs, static schema, multiple auth options | | **MCP Server Action** | Model Context Protocol | AI-native, dynamic tool discovery, streamable (Preview) | | **Copilot Studio Action** | Power Platform Connectors | 1400+ prebuilt connectors, visual designer, citizen-developer friendly | --- ### 3. Copilot APIs: Embedding Copilot in Your Apps **Direction: Your Apps → Copilot** Copilot APIs let you programmatically access Copilot's capabilities from your own applications. **Available APIs:** | API | Status | Purpose | |-----|--------|---------| | Retrieval API (RAG) | GA | Query Microsoft Graph with AI-enhanced retrieval | | Search API | Preview | Semantic search across M365 content | | Chat Completions API | Preview | Conversational AI with M365 context | | Meeting Transcript API | GA | Access meeting transcriptions | | Meeting Insights API | Preview | Extract insights from meetings | **Best for:** - Building custom applications that leverage Copilot - Integrating AI capabilities into existing line-of-business apps - Creating specialized user experiences --- ## Common Pitfalls and How to Avoid Them ### Pitfall #1: Using Connectors When You Need Actions **Symptom:** "I connected my CRM data, but Copilot can't create new records." **Solution:** Connectors are read-only. To write data back, create an Agent with Actions that call your CRM's API. ### Pitfall #2: Building Custom Engines When Declarative Would Suffice **Symptom:** Spending months building a custom AI engine for a simple Q&A bot. **Solution:** Start with Declarative Agents. They deploy faster and Microsoft handles the AI infrastructure. Only go custom when you need specific models or complex orchestration. ### Pitfall #3: Ignoring Data Governance **Symptom:** Copilot surfaces sensitive data to unauthorized users. **Solution:** Connectors and Agents respect Microsoft 365 permissions. Ensure your data sources have proper access controls *before* connecting them. ### Pitfall #4: Overlooking the Actions-in-Agents Requirement **Symptom:** "I built an OpenAPI action but can't find it in Copilot." **Solution:** Actions must be deployed within an Agent. Create a Declarative Agent, add your action, and deploy the agent to Teams or M365. --- ## Decision Framework: Choosing the Right Approach ``` Do you need Copilot to access external data? ├── Yes → Use Graph Connectors │ └── Do you also need to write back? │ └── Yes → Add an Agent with Actions └── No ↓ Do you need Copilot to perform tasks? ├── Yes → Build an Agent │ ├── Simple tasks, quick deployment → Declarative Agent │ └── Complex logic, custom AI → Custom Engine Agent └── No ↓ Do you need AI in your own app? ├── Yes → Use Copilot APIs └── No → You might not need extensibility! ``` --- ## Getting Started ### Tools You'll Need | Tool | Purpose | Skill Level | |------|---------|-------------| | **Copilot Studio** | Visual agent builder | Low-code | | **M365 Agents Toolkit** | VS Code extension for agent development | Pro-code | | **Kiota CLI** | OpenAPI client generation | Pro-code | | **Graph API** | Custom connector development | Pro-code | ### Recommended Learning Path 1. **Start with Copilot Studio** to understand agent concepts visually 2. **Build a Declarative Agent** with knowledge sources 3. **Add an OpenAPI Action** to perform a simple task 4. **Explore MCP** for more dynamic scenarios 5. **Graduate to Custom Engine Agents** when you hit limitations --- ## Conclusion Microsoft 365 Copilot extensibility is powerful but requires understanding the architecture. Remember: - **Connectors** bring data IN (read-only) - **Agents** enable bidirectional interaction with Actions - **APIs** let you bring Copilot OUT to your apps - **Actions only work inside Agents**—this is the #1 gotcha Start simple, validate your approach, and scale from there. The extensibility landscape is evolving rapidly, with MCP and new APIs in preview that will expand what's possible. --- *Have questions about Copilot extensibility? The patterns and pitfalls shared here come from real-world implementation experience. Your mileage may vary, but understanding these fundamentals will set you on the right path.* --- title: "The Future of Copilot Agents - An Agent-Centric Workplace" url: https://bisser.io/the-future-of-copilot-agents-an-agent-centric-workplace/ markdown: https://bisser.io/the-future-of-copilot-agents-an-agent-centric-workplace/index.md date: 2025-10-27 lastmod: 2025-10-27 tags: AI, Copilot, Generative AI, Gen AI, Agents, Microsoft 365 Copilot categories: AI, Copilot description: "The workplace is evolving from prompt-based assistants to an agent-centric model, where specialized digital teammates pursue goals autonomously. Microsoft’s ecosystem—from SharePoint to Azure AI Foundry—empowers everyone to build agents that seamlessly extend Microsoft 365 Copilot. What began wit..." status: current reading_time_min: 4 --- # Introduction Today we live in an era of constant change. Technology changes, mindset changes and vision changes. It started out with a bunch of Generative AI tools fueled by LLMs. These AI Tools now do not only offer the access to LLMs itself, but also give you the chance of creating and using agents. These agents give you two extra-assets: - Consistency - Scope My friend [Thomy](https://thomy.tech/) wrote about the topic of [AI Agents at Process Handovers: Where Consistency Matters More Than Intelligence](https://www.linkedin.com/pulse/ai-agents-process-handovers-where-consistency-matters-thomas-g%C3%B6lles-c0wwf/?trackingId=rsfBNBFP7E%2FlwsZHCPbMmQ%3D%3D), so if you want to know about why consistency matters, please read that first. But with the introduction of agents, we also started shifting into an "Agent-Centric Workplace". And I started to feel this myself rather quick: For every platform, for every service and for every use case, an agent could be built to assist me throughout my workday. To me, using agents goes beyond using Copilot. It's more a concept of delegating work to my digital teammates = agents. And with all the recent announcements and udpates published by Microsoft around topics like autonomous agents, I do not think that this concept is going to vanish in the future. More likely, we will see the transition going from prompt-based assistants to goal/task-oriented, multi-step reasoning systems more and more over the course of the next weeks, months and years. ## Ecosystem The Microsoft platforms offers a sophisticated ecosystem when it comes to building (and yes you may think why is he not writing "developing" instead of building, but we'll come to that in a second) agents of many kinds. You can build agents in a variety of places with a variety of tools like: - SharePoint - Copilot Studio lite - Copilot Studio full - Microsoft Fabric - M365 Agents Toolkit & SDK - Azure AI Foundry - Agent Framework - ... And all of these platforms and frameworks target a specific agent builder audience, from low-code to pro-code. Eventually I will write another blog post, or even a series, about which platforms should be leveraged by which audience. But the good news here is, that there is at least one tool or platform for everyone, no matter which skills you have when it comes to building or developing an agent. And many of the agents built with these platforms can then be again consumed from one central place: Microsoft 365 Copilot. Therefore, the platform will likely evolve and governance aspects will also be covered more and more as more and more people will not only be using agents, but will also be building agents, either for their personal use, for their team or for the whole organization. ## Multi-Agent Collaboration With the variety of tools and frameworks, the necessacity of introducing a concept called "Multi-Agent Collaboration" grew. Satya Nadella once said in a keynote of a big conference that "Copilot is the UI for AI". Thinking about this message, I thought that this is the beginning of the Multi-Agent Collaboration era, where Microsoft 365 Copilot itself is also an agent, having the skills of orchestrating and collaborating with other agents. But for me personally, hearing this message was not the first time for me personally to think of multi-agent scenarios. Looking back at my blog post on [Microsoft Build 2019 updates on Conversational AI](https://bisser.io/microsoft-build-2019-updates-on-conversational-ai/), the concept of **Skills** was introduced. And this to me was the first step towards a multi-agent collaboration era, because skills in the realm of the Microsoft Bot Framework where chatbots itself, which other chatbots would call to solve specific tasks. ### Human-Agent Collaboration Patterns In the future, we might see a shift from "Copilot in apps" to "Copilot across work" as this may be something that people need more urgent. In my idea, I think an agent like a "Project Planning Agent" might be more helpful when integrated in all of my work apps, than an agent like the Copilot in Word. Because I want to perfectly do my project planning, which involves multiple platforms and services, so this agent should be accessible and integrated in all of the places where I do my work and is still specifically tailored to one specific goal (here we are again at consistency and scope). This shit may also require us to rethink productivity rituals, like how do we manage meetings or how do we do reporting or how do we share knowledge when agents are our digital teammates, as they also need to be integrated into these rituals. ## Future look With things like MCP, multimodal models and agents and the convergence of local and cloud intelligence, I do think that agents will play an important role in our future workplace. In the future we might not be talking "Copilot Adoption" but "Agent lifecycle management", as I strongly believe that the introduction of "goal-oriented agents" will be a huge boost in the enterprise, to increas both our productivity as well as the quality we deliver our work with. Therefore I encourage everyone reading this, to experiment with agents and especially with multi-agent scenarios as this will likely hit us sooner or later and furthermore, think about this: "What's your vision for the agentic workplace?" and let me know what's your opinion! --- title: "Exploring Autonomous Agent Capabilities with Microsoft Copilot Studio" url: https://bisser.io/exploring-autonomous-agent-capabilities-with-microsoft-copilot-studio/ markdown: https://bisser.io/exploring-autonomous-agent-capabilities-with-microsoft-copilot-studio/index.md date: 2025-03-29 lastmod: 2025-03-29 tags: AI, Copilot, Generative AI, Gen AI categories: AI, Copilot description: "Examining how Microsoft Copilot Studio's preview release of autonomous agent capabilities lays a basic yet promising foundation—highlighting its core architecture, current functionalities, and future potential for automating enterprise workflows." status: current reading_time_min: 3 --- # Introduction Microsoft is pushing the boundaries of how business processes can be automated with its **Copilot Studio**—a cloud-based, low-code platform that empowers organizations to build AI agents. In its latest release, Microsoft has introduced autonomous agent capabilities aimed at enabling agents to proactively respond to events, orchestrate tasks, and integrate seamlessly with enterprise data sources. However, while these functionalities open exciting prospects, in my opinion, they currently feel quite basic—likely reflecting their preview status. ## Autonomous Agent Capabilities at a Glance With Copilot Studio, Microsoft now provides tools to build autonomous agents that can: - **Monitor & React:** Automatically respond to business signals or triggers to initiate tasks. - **Execute Business Processes:** Leverage AI orchestration to run rule-based workflows and automate repetitive tasks. - **Integrate with Data Sources:** Connect to Microsoft Graph, Dataverse, and other connectors to pull in context. - **Enhance Productivity:** Offer a low-code way for teams to extend Microsoft 365 Copilot with personalized AI agents. ## Fundamental Architecture Diagram A core element of understanding autonomous agents in Copilot Studio is their underlying architecture. Below is a diagram which displays a visual representation of the system: {{< mermaid >}} graph LR; %% Trigger: Automated trigger A["Trigger
(Recurrence/Power Platform Connector)"] %% Core Components within Copilot Studio subgraph Core["Copilot Studio Orchestrator"] B1["Understand trigger"] B2["Analyze trigger input"] B3["Build plan"] end A --> Core %% Backend: Data and Knowledge Sources subgraph Data["Data & Knowledge Sources"] D1["Add grounding data"] D2["Execute actions & plan"] D3["Generate output"] end Core --> Data %% User: User interface subgraph User["User interface"] U1["Send output"] end Data --> User {{< /mermaid >}} {{< figure src="/images/080-overview-diagram.png" >}} *Figure 1: Fundamental Architecture for Autonomous Agents in Microsoft Copilot Studio* This diagram illustrates how user interactions are processed through Copilot Studio, which utilizes an orchestration engine to understand the trigger and inputs along with building and executing a plan. The agents then access context-rich data from secure sources to generate responses. ## The Preview Nature: Opportunities and Limitations While the preview release is promising, the current autonomous agent capabilities appear somewhat basic: - **Limited Customization:** The workflow options and decision-making granularity are still evolving. - **Early-Stage Integrations:** Integrations with external systems work at a basic level—expect enhancements with additional connectors and refinements. - **Dependence on User Feedback:** Early adopters find that while the automation framework offers efficiencies, further adjustments and human oversight are needed to handle complex real-world scenarios. These observations suggest that the preview is an important first step, setting the stage for further refinements as Microsoft iterates based on customer feedback. ## Looking Ahead Microsoft’s vision for transforming business processes with AI is clear. As additional features and deeper integrations are developed, we can expect: - More advanced decision-making capabilities. - Enhanced automation for complex workflows. - Greater interoperability across the Microsoft ecosystem. Even though the functionality now might seem basic, it represents a foundational step towards a future where AI agents can autonomously handle a wide array of business tasks with minimal human intervention. --- What do you think about the current state of autonomous agent capabilities in Copilot Studio? Do you see this basic functionality as laying the groundwork for a more advanced, fully autonomous future? --- title: "Add deep reasoning to an Copilot Studio agent" url: https://bisser.io/add-deep-reasoning-to-an-copilot-studio-agent/ markdown: https://bisser.io/add-deep-reasoning-to-an-copilot-studio-agent/index.md date: 2025-03-26 lastmod: 2025-03-26 tags: AI, Copilot, Generative AI, Gen AI categories: AI, Copilot description: "Enabling deep reasoning on a Microsoft Copilot Studio agent and use it in Microsoft 365 Copilot" status: current reading_time_min: 1 --- ## Setup You obviously need to create or already have an agent in Microsoft Copilot Studio. If you don't have an agent already follow [this tutorial](https://learn.microsoft.com/en-us/microsoft-copilot-studio/fundamentals-get-started?context=%2Fmicrosoft-365-copilot%2Fextensibility%2Fcontext&tabs=web) to create a new custom engine agent in Copilot Studio. After you have created an agent go to your settings page of the agent and then under "Generative AI" you can enable the checkmark on "Use deep reasoning models": {{< figure src="/images/079-mcs-enable-deep-reasoning.png" >}} When you now send a prompt to your agent you should see that the agent is using deep reasoning to generate the answer: {{< figure src="/images/079-mcs-deep-reasoning-test.png" >}} What's interesting is that you can also see what the agent does take into account when doing the reasoning: {{< figure src="/images/079-mcs-deep-reasoning-info.png" >}} As you can now also add your custom engine agents built with Copilot Studio to your Microsoft 365 Copilot you can use this method to add deep reasoning to Microsoft 365 Copilot: {{< figure src="/images/079-mcs-deep-reasoning-m365.png" >}} ## Considerations This feature is currently in preview so be careful when to enable this (maybe not the best idea to use this for all production agents already). Another requirement is that your agent needs to be created in a US-based Power Platform environment and the agent's language should be English, otherwise you'll see the setting, but the agent will not use deep reasoning unfortunately. --- title: "Introducing Agent Instructor - A VS Code extension for managing agent instructions" url: https://bisser.io/introducing-agent-instructor-a-vs-code-extension-for-managing-agent-instructions/ markdown: https://bisser.io/introducing-agent-instructor-a-vs-code-extension-for-managing-agent-instructions/index.md date: 2025-02-20 lastmod: 2025-02-20 tags: AI, Copilot, Generative AI, Gen AI categories: AI, Copilot description: "A powerful VS Code extension designed to help developers create, analyze, and refine instructions for declarative AI agents." status: current reading_time_min: 1 --- ## Agent Instructor Agent Instructor is a powerful Visual Studio Code extension designed to help developers create, analyze, and refine instructions for declarative AI agents. This tool is essential for ensuring clarity and effectiveness in agent instructions, making it easier to build robust AI solutions. ### Key Features: **Analyze Instructions:** - Clarity Score: Evaluates existing instructions and provides a clarity score (0-100). - Ambiguity Detection: Identifies ambiguous phrases and offers specific improvement suggestions. - One-Click Corrections: Allows users to apply suggested improvements with a single click. - Visual Analysis: Displays analysis results with intuitive charts. **Generate Instructions:** - Interactive Input: Facilitates the creation of comprehensive instructions through interactive agent description input. - AI-Powered Generation: Automatically generates well-structured and formatted instructions. ## Installation Head over to the [VS Code Marketplace](https://marketplace.visualstudio.com/items?itemName=Stephan-Bisser.agent-instructor) to install the extension from there or search for "Agent Instructor" in the VS Code extension store. Feel free to leave feedback as I'm interested in your opinion! --- title: "Copilot for Microsoft 365 Series - 7 - Copilot Agent Building Options" url: https://bisser.io/copilot-for-microsoft-365-series-7-copilot-agent-building-options/ markdown: https://bisser.io/copilot-for-microsoft-365-series-7-copilot-agent-building-options/index.md date: 2025-01-21 lastmod: 2025-01-21 tags: AI, Copilot, Generative AI, Gen AI categories: AI, Copilot description: "A comparison infographic on the various options you have when building Microsoft 365 Copilot Agents, from no-code to low-code to pro-code." status: current reading_time_min: 2 --- ## Introduction As of now, there are a lot of options for building agents for Microsoft 365 Copilot and related services. If you are a heavy SharePoint user you can build agents which are grounded into your SharePoint knowledge with only a few clicks right within the SharePoint UI. If you are a low code maker, you can use either the Copilot Agent Builder or Microsoft Copilot Studio to build declarative agents which can then be surfaced through the Microsoft 365 Copilot Chat UX or Microsoft Teams. And if you are a pro developer you can build either declarative agents using Teams Toolkit for Visual Studio Code or Visual Studio or you can build a custom engine agent using whatever tech stack you like. That may range from integration services such as the Teams AI Library to the Microsoft 365 Agents SDK to Azure AI Foundry or any other API you may want to consume. As there are many options for building agents, I tried to visualize all of them into one single infographic: {{< figure src="/images/077_Copilot_Agent_Building_Options.png" >}} ## Conclusion As the infographic shows, you have a lot of options to choose from when it comes to building agents for Microsoft 365 Copilot. Feel free to use it for internal use in your presentations or documents. The only point I ask you to do is to credit me and link this article to your documents where you use it. Please do not sell this as your own 😉 --- title: "Copilot for Microsoft 365 Series - 6 - Extensibility options Wave 2 updates" url: https://bisser.io/copilot-for-microsoft-365-series-6-extensibility-options-wave-2-updates/ markdown: https://bisser.io/copilot-for-microsoft-365-series-6-extensibility-options-wave-2-updates/index.md date: 2024-10-01 lastmod: 2024-10-01 tags: AI, Copilot, Generative AI, Gen AI categories: AI, Copilot description: "Everything you need to know about the different ways of extending Copilot for Microsoft 365 including agents, plugins and connectors which have been announced/updated after the Copilot Wave 2 event." status: current reading_time_min: 2 --- ## Introduction In early 2024 I released the first version of the [Microsoft 365 Copilot extensiblilty options guide](https://bisser.io/copilot-for-microsoft-365-series-2-extensibility-options/). Since then a lot has changed, especially the fact that Microsoft announced a at Build the capability of not only building Connectors and Plugins, but also [Agents](https://learn.microsoft.com/en-us/microsoft-365-copilot/extensibility/overview-declarative-agent). This path was further strengthened with the[ Microsoft 365 Copilot Wave 2 announcements](https://www.youtube.com/watch?v=Gz0jPfCuvsM) in September 2024. Despite many other updates and announcements like Copilot Pages and others, the agentic capabilities within Copilot is one of the biggest updates which lets developers not only extend Copilot using plugins or connectors, but now you can build declarative agents which combine plugins and connectors into one single solution which can then be used either seperately or within the Microsoft 365 Copilot experiences. As there where many updates and changes, I modified the initial version of my decision guide, which can be found in the following infographic: {{< figure src="/images/076_Copilot_Extensibility_which_option_when_v2.png" >}} ## Conclusion As the infographic shows, you have a lot of options to choose from when it comes to finding the suitable approach for extending Microsoft 365 Copilot. And when the Agent Builder functionality is rolled out, I will certainly add this to that infographic as well. Feel free to use it for internal use in your presentations or documents. The only point I ask you to do is to credit me and link this article to your documents where you use it. Please do not sell this as your own 😉 --- title: "Copilot for Microsoft 365 Series - 5 - Hidden prerequisites" url: https://bisser.io/copilot-for-microsoft-365-series-5-hidden-prerequisites/ markdown: https://bisser.io/copilot-for-microsoft-365-series-5-hidden-prerequisites/index.md date: 2024-03-25 lastmod: 2024-03-25 tags: AI, Copilot, Generative AI, Gen AI categories: AI, Copilot description: "All Copilot for Microsoft 365 requirements at a glance" status: current reading_time_min: 3 --- ## Introduction In this post, we will delve into the hidden prerequisites and requirements for using Copilot for Microsoft 365. We aim to provide a comprehensive guide that will help you understand what you need to get started and ensure a smooth operation of Copilot within your Microsoft 365 environment. Let's uncover these hidden prerequisites together. ## Copilot requirements The requirements and prerequisites for Copilot for Microsoft 365 are very well documented [here](https://learn.microsoft.com/en-us/microsoft-365-copilot/microsoft-365-copilot-requirements). Of course I asked Copilot to generate a list of requirements, so here it is: * Base License: Users need a prerequisite base license, detailed in the service description guide. * Microsoft 365 Apps: Deployment of Microsoft 365 Apps is necessary. * Third-party Cookies: For online versions of Word, Excel, and PowerPoint, third-party cookies must be enabled. * Privacy Settings: Review Microsoft 365 Apps privacy settings as they affect Copilot features. * Microsoft Entra ID: Users require Microsoft Entra ID accounts, which can be added or synced via the admin center. * OneDrive Account: Certain features like file restore need users to have a OneDrive account. * Outlook: Copilot works with the new Outlook in preview, and settings sync is required for integratio5. * Teams: Configuration of Teams settings is essential for Copilot functionality in Teams. * Teams Phone: For referencing meeting content, transcription or recording needs to be enabled. * Microsoft Loop & Whiteboard: Enablement of Loop and Whiteboard for tenant use is required. * Update Channels: Copilot is supported in all update channels, except for Semi-Annual Enterprise Channel. * Network Connectivity: Aligning with Microsoft 365 network connectivity principles is recommended. * WebSocket Connections: Allow WebSocket connections for Copilot experiences that require them. In terms of licensing, Copilot for Microsoft 365 is available as an add-on plan with one of the following licensing prerequisites: * Microsoft 365 E5 * Microsoft 365 E3 * Office 365 E3 * Office 365 E5 * Microsoft 365 Business Standard * Microsoft 365 Business Premium * Microsoft 365 A5 for faculty * Microsoft 365 A3 for faculty * Office 365 A5 for faculty Please note that these requirements are nearly identical to the requirements for using Microsoft 365 Apps due to the integration of Microsoft Copilot for Microsoft 365 and Microsoft 365 Apps. **But the list of requirements was missing one crucial thing: Enablement of the Office Feature Updates task from the scheduled tasks on the Windows client. If the execution of the following 2 tasks is blocked, then Copilot will not work on the client and in the local versions of the Office apps:** * **Office Feature Updates** * **Office Feature Updates Logon** ## Conclusion Make sure to check out the [Copilot requirements](https://learn.microsoft.com/en-us/microsoft-365-copilot/microsoft-365-copilot-requirements) from time to time to see if anything has changed. Shout-out to my colleague Gottfried Jocham, for detecting the need for the scheduled task. --- title: "Copilot for Microsoft 365 Series - 4 - Dall-E 3 sample plugin" url: https://bisser.io/copilot-for-microsoft-365-series-4-dall-e-3-sample-plugin/ markdown: https://bisser.io/copilot-for-microsoft-365-series-4-dall-e-3-sample-plugin/index.md date: 2024-03-15 lastmod: 2024-03-15 tags: AI, Copilot, Generative AI, Gen AI categories: AI, Copilot description: "A short guidance on how to create a Copilot for Microsoft 365 plugin including a sample" status: current reading_time_min: 3 --- ## Introduction A while ago I created my first plugin for Copilot for Microsoft 365. Therefore, I would like to take the opportunity to share my experience and insights. You can find the working plugin sample in my [GitHub repository](https://github.com/stephanbisser/Copilot-for-M365-Dall-E-Plugin). ## The setup I used the [Teams Toolkit for Visual Studio Code](https://learn.microsoft.com/en-us/microsoftteams/platform/toolkit/teams-toolkit-fundamentals) to develop my plugin. You also need to make sure that you fulfill all the requirements mentioned [here](https://learn.microsoft.com/en-us/copilot-plugins/get-started#prerequisites). Additionally, you need to deploy an Azure Open AI resource in your Azure subscription. Make sure sure to choose an Azure region which offers Dall-E models from the [list of available regions](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models#dall-e-models), otherwise you cannot deploy a Dall-E model. ## Developing the plugin When cloning my repo, make sure that you create a file called `.localConfigs` in the root of your plugin's folder and add the following content to that file along with the correct values for those parameters: ``` BOT_ID= BOT_PASSWORD= SECRET_AOAI_KEY= AOAI_ENDPOINT= ``` The next important part is your manifest. The manifest is crucial as Copilot uses it to determine what your plugin will do and how it can be used as well as when it should be triggered. Therefore, make sure to add meaningful descriptions to your app, your commands, and your parameters: {{< figure src="/images/074_manifest_sample.png" >}} Then within the searchApp.ts you will need to call your Azure Open AI resource API to generate an image and retreive it: ```ts public async handleTeamsMessagingExtensionQuery( context: TurnContext, query: MessagingExtensionQuery ): Promise { const searchQuery = query.parameters[0].value; const attachments = []; //const apiKey = ""; // Insert your Azure Open AI API Key here const service = new DalleService(apiKey, endpoint, searchQuery); switch (query.commandId ) { case 'generateImage': console.log('Trying to generate image using Dall-E'); let response = await service.makeApiCall(); const template = new ACData.Template(imageCard); const card = template.expand({ $root: { prompt: response.data[0].revised_prompt, imageUrl: response.data[0].url, }, }); const preview = CardFactory.heroCard(response.data[0].revised_prompt); console.log('Revised Prompt: ', response.data[0].revised_prompt); console.log('Image URL: ', response.data[0].url); const attachment = { ...CardFactory.adaptiveCard(card), preview }; attachments.push(attachment); return { composeExtension: { type: "result", attachmentLayout: "list", attachments: attachments, }, }; break; } return { composeExtension: { type: "result", attachmentLayout: "list", attachments: attachments, }, }; } ``` To send the image back to the user, simply use an Adaptive Card, which is used to render the image: ```json { "$schema": "http://adaptivecards.io/schemas/adaptive-card.json", "type": "AdaptiveCard", "version": "1.5", "body": [ { "type": "TextBlock", "text": "${prompt}", "wrap": true, "size": "Large" }, { "type": "Image", "url": "${imageUrl}", "size": "Large" }, { "type": "ActionSet", "actions": [ { "type": "Action.OpenUrl", "title": "Open Image", "url": "${imageUrl}" } ] } ], "previewCard": { "contentType": "application/vnd.microsoft.card.hero", "content": { "title": "${prompt}", "text": "${prompt}" } } } ``` And tha's it, you've built your first plugin. In VS Code you can now hit F5 and test it right within Copilot in Teams! ## Conclusion Plugins play a crucial role in enhancing the Copilot ecosystem. They enable the expansion of Copilot’s inherent abilities and facilitate integration with your line-of-business applications or external platforms. Therefore, it’s important to tailor Copilot with plugins that suit your specific requirements and scenarios. However, it’s equally important to provide detailed and accurate descriptions for these plugins to ensure Copilot recognizes and utilizes them effectively. --- title: "Copilot for Microsoft 365 Series - 3 - Technical architecture" url: https://bisser.io/copilot-for-microsoft-365-series-3-technical-architecture/ markdown: https://bisser.io/copilot-for-microsoft-365-series-3-technical-architecture/index.md date: 2024-03-01 lastmod: 2024-03-01 tags: AI, Copilot, Generative AI, Gen AI categories: AI, Copilot description: "Wondering how Copilot for Microsoft 365 is working technically? This post describes the technical architecure briefly" status: current reading_time_min: 2 --- ## Introduction It's a bit confusing how Copilot for Microsoft 365 actually works as there are a lot of components involved. Therefore, this short post should give you a glimpse on how the inner parts of Copilot work. ## How Copilot works As you can see in the infographic below, the Copilot system consists of multiple components: - Copilot Service - Microsoft Graph - Semantic Index - Plugin catalog - Plugins The most important thing to understand, is that currently plugins are not available everywhere. At the moment you can use plugins here: - Copilot (previously called "M365 Chat") - Microsoft Teams - Microsoft 365 Portal - Edge (Although here is only the "Web content" plugin available) In all other applications like Word, Excel, PowerPoint and so on, plugins are not yet available. So therefore, you can only use plugins where you have a more chat-based interface where Copilot assists you at the moment. The way Copilot plugins work is as follows: 1. User's prompts in Microsoft 365 Apps are sent to Copilot 2. Copilot determines user’s intent to decide what actions should follow next 3. If Microsoft 365 data is required, Copilot grounds prompt before forwarding to LLM & after receiving LLM response using Graph & Semantic Index 4. If non-Microsoft 365 data is required, Copilot fetches available plugins & connectors to decide where to retrieve information from. If user’s intent is to take actions in a 3rd party system with a plugin available, Copilot uses plugin to execute plan {{< figure src="/images/073_copilot_technical_architecture.png" >}} Therefore, it is vital that your plugin description as well as your command & parameter descriptions in your plugin's manifest files are as precise as they can be. Only then Copilot knows which plugins to pick when and how to treat them accordingly so that user's will benefit from your plugins. ## Conclusion Plugins are an essential part of the whole Copilot ecosystem as they allow you to extend the skills Copilot has out of the box and connect it to your LOB apps or 3rd party platforms. So make sure to extend Copilot to fit your needs and use cases, but remember to be precise on plugin descriptions for Copilot to pick it up! Feel free to use the infographic for internal use in your presentations or documents. The only point I ask you to do is to credit me and link this article to your documents where you use it. Please do not sell this as your own 😉 --- title: "Copilot for Microsoft 365 Series - 2 - Extensibility options" url: https://bisser.io/copilot-for-microsoft-365-series-2-extensibility-options/ markdown: https://bisser.io/copilot-for-microsoft-365-series-2-extensibility-options/index.md date: 2024-02-27 lastmod: 2024-02-27 tags: AI, Copilot, Generative AI, Gen AI categories: AI, Copilot description: "Everything you need to know about the different ways of extending Copilot for Microsoft 365 including plugins and connectors." status: current reading_time_min: 2 --- ## Introduction As I now used Copilot for Microsoft 365 quite some time and also worked in various Copilot introduction projects with customers, I found out, that the out-of-the-box experience, for me, wasn't enough. Therefore I did a deep dive into the [documentation](https://learn.microsoft.com/en-us/microsoft-365-copilot/extensibility) on how to extend Copilot for Microsoft 365. In the docs, there is a [decision guide](https://learn.microsoft.com/en-us/microsoft-365-copilot/extensibility/decision-guide) which visualizes the ways of extending Copilot for Microsoft 365 and in general you have 2 options: - Extending the data source Copilot has access to - This basically is just about giving Copilot access to more than "just" Microsoft 365 data sources like SharePoint, OneDrive, Exchange, and so on but also including 3rd party data stores like other SaaS platforms, on premises data sources like file shares or data stored in Azure. But choosing this option does not mean, that Copilot can bidirectional communicate with those other data stores. This option is a one-way option which only allows data to flow into the Microsoft 365 index but not the other way around. - Extending the Copilot functionality - This option lets you extend the features Copilot offers in various situations and it also allows you to integrate Copilot with other services (3rd party platforms, custom LOB apps and so on) in a way that it can retreive data from but also writes data into those services, in other words a bidirectional communication flow. So when choosing this option, you can allow users to communicate with other services outside of the Microsoft 365 ecosystem for instance through Copilot to retreive and ingest data. And for those two options you basically have muliple ways of implementing that kind of extensibility options. But in the decision guide, there was one thing missing which is essential when talking about Copilot extensibility: Copilot Studio. Therefore, I created a modified version of the decision guide, which can be found in the following infographic: {{< figure src="/images/072_Copilot_Extensibility_which_option_when.png" >}} ## Conclusion As the infographic shows, you have a lot of options to choose from when it comes to finding the suitable approach for extending Copilot for Microsoft. In the next blog post(s) we'll take a look on how to deal with all those options in greater detail! Feel free to use it for internal use in your presentations or documents. The only point I ask you to do is to credit me and link this article to your documents where you use it. Please do not sell this as your own 😉 --- title: "Copilot for Microsoft 365 Series - 1 - The different flavours of Copilot" url: https://bisser.io/copilot-for-microsoft-365-series-1-the-different-flavours-of-copilot/ markdown: https://bisser.io/copilot-for-microsoft-365-series-1-the-different-flavours-of-copilot/index.md date: 2023-11-21 lastmod: 2023-11-21 tags: AI, Copilot, Generative AI, Gen AI categories: AI, Copilot description: "A fresh start of something new. I would like to take you on a journey about things I learnt while using Copilot for Microsoft 365" status: current reading_time_min: 4 --- ## Introduction It has been a while since I wrote my last blog post (more than half a year - time flies). I had a lot of things to do while I "paused" my blogging game, but now I wanted to start sharing some insights again. And this time about a topic which is everywhere and everything: Copilot. I'm using various Copilots for quite some time now and wanted to share my learnings I had while using it. ## Outline To be precise: What I will share is my experience with Copilot. You might now think, which Copilot, right? I know there are a ton of Copilots out there. However, I will focus on Copilot, Copilot and Copilot 😉 But to make you understand better which Copilots, the following Infographic should help you understand which Copilots I'm using and for which Copilots you can expect to gain information throughout this blog series. {{< figure src="/images/071_copilot_offering.png" >}} Let me explain you the differences of those 3 briefly. ## Copilot [Copilot](https://bing.com/chat) (or formerly know as "Bing Chat Enterprise") is the equivalent to ChatGPT but baked into Bing and moreover, into Microsoft's Edge browser. I use it quite some time now as it's a great companion while browsing through websites and reading information as it provides concise summaries of websites within a few seconds. On top of that I used it as tool to brainstorm and generate ideas, but I replaced that use case now with something different (scroll down to see what I'm using now). ## Custom Copilot Well, what a ride. For those of you who don't know, to me Custom Copilot has quite some history. As I see it as a successor to not only Power Virtual Agents, but to all other related tech in Microsoft's Conversational AI space, like Bot Framework SDK, Bot Framework Composer, Azure Bot Service and many more. Copilot studio nowadays lets you do two main things: - Extend 1st party Copilots (like the *[Copilot for Microsoft 365](https://adoption.microsoft.com/en-us/copilot/)*) - Build custom Copilots (like a company HR copilot) But I personally am using custom copilots for years now and I have started building our very first internal copilot (or chatbot as it was called back in the days) in January 2020 using the Microsoft Bot Framework SDK and Azure Bot Service in addition to LUIS and QnA Maker. And this technology stack has evolved over time to include [Bot Framework Composer](https://aka.ms/bfcomposer), Power Virtual Agents as well as Azure Open AI. I will definetely spend at least one blog post in this series going into greater detail when it comes to Copilot Studio. ## Copilot for Microsoft 365 Microsofot 365 Copilot or Copilot for Microsoft 365 which is the new name for it is now my personal copilot for quite some time actually as I had the chance to get it before GA on November 1st. So I played around with it and use it now an a daily basis. The use cases where Copilot comes in handy are different. But here is a list of use cases I personally think Copilot is a great tool: - Support me during the complete Meeting lifecycle - Help me draft notes, agenda, discussion points before a meeting - Stay on track what has been discussed and agreed on during a meeting - Work on my action items and tasks after a meeting - Summarize long email conversations in Outlook - Especially when there is a lot of noise in my Outlook inbox, Copilot helps me stay up to date what the main points within an endless email back-and-forth are - Be creative - This is probably the most relevant part for me personally, as I now have a virtual assistant who joins me being creative whenever I need to. No matter if I need to create a new slide or a whole slide deck for a customer pitch or conference session or if I need to come up with something new in my work tasks - Copilot helps me make my creativity process more efficient and moreover, less time consuming. And if I don't know if and how Copilot can support me work on a specific task, I just ~~ask~~prompt. ## Conclusion As you can see I've got one or the other story and opinion to share so stay tuned to learn more about my journey with Copilot(s)! And as usual if you have questions or topics you want me to cover, just ~~ping~~prompt me. --- title: "Bot Framework Composer Series - 5 - Proactive Messages" url: https://bisser.io/bot-framework-composer-series-5-proactive-messages/ markdown: https://bisser.io/bot-framework-composer-series-5-proactive-messages/index.md date: 2023-03-15 lastmod: 2023-03-15 tags: AI, Bot Framework, Teams Development categories: AI, Bot Framework description: "The 5th post in this series covers how to extend Bot Framework Composer to support proactive messages" status: current reading_time_min: 5 --- ## Introduction In this post I will show you how to extend Bot Framework Composer to support proactive messages. I will also show you how to use the extension to send proactive messages from a web application. The goal here is to use the feature of [proactive messages](https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-howto-proactive-message?view=azure-bot-service-4.0&tabs=csharp) to send notifications to users in Microsoft Teams from a web application or API. ## The implementation What we need is basically any chatbot build with Bot Framework Composer. So you can either go ahead and create a new one (it doesn't matter which template you choose) or you can use a bot which you already built using Composer. I will use the bot I created in the previous post in this series. You can find the code for the bot [here](https://bisser.io/bot-framework-composer-series-4-send-a-user-profile-card/). There are basically 3 things we need to do to extend Composer to support proactive messages: - We need to add an additional controller to the bot called NotifyController which will be used to manage and send the proactive messages - We need at least one Adaptive Card to be used within the proactive message towards the user ### NotifyController The NotifyController is the controller which will be used to send the proactive messages. It will be called from a web application or API to send the message. The controller will be added to the bot project and will be called from the web application or API. The controller will be called with a JSON payload which will contain the following information: ```json { "botId": "28:botid", "userId": "29:userid", "conversationId": "19:conversationid", "cardName": "oppCard.json" } ``` The controller will then use the information from the JSON payload to send the message to the user. Therefore the controller to be added into the _Controllers_ folder of the bot project will look like this: ```C# using System.Threading.Tasks; using Microsoft.AspNetCore.Mvc; using Microsoft.Bot.Builder; using Microsoft.Bot.Builder.Integration.AspNet.Core; using Microsoft.Bot.Schema; using Microsoft.Extensions.Configuration; using System; using System.Collections.Concurrent; using System.Collections.Generic; using System.IO; using System.Linq; using System.Net; using System.Text; using System.Threading; using System.Web; using Microsoft.Bot.Builder.Teams; using Microsoft.Bot.Connector; using Microsoft.Bot.Connector.Authentication; using Microsoft.Bot.Schema.Teams; using Microsoft.Extensions.Logging; using Newtonsoft.Json; using Newtonsoft.Json.Linq; namespace ErdwinDemo.Controllers { [Route("api/notify")] [ApiController] public class NotifyController : ControllerBase { private readonly IBotFrameworkHttpAdapter _adapter; private readonly IBot _bot; private string _appId; private string _appSecret; private string _serviceUrl; private string _conversationId; public NotifyController(IBotFrameworkHttpAdapter adapter, IBot bot, IConfiguration configuration) { _adapter = adapter; _bot = bot; _appId = configuration["MicrosoftAppId"]; // The Microsoft App Id of your Bot _appSecret = configuration["MicrosoftAppPassword"]; // The Microsoft App Password of your Bot _serviceUrl = "https://smba.trafficmanager.net/emea/"; } [HttpPost] public async Task Post([FromBody] string content) { MicrosoftAppCredentials.TrustServiceUrl(_serviceUrl); var connectorClient = new ConnectorClient(new Uri(_serviceUrl), new MicrosoftAppCredentials(_appId, _appSecret)); dynamic json = JsonConvert.DeserializeObject(content); var bot = json["botId"].Value; var user = json["userId"].Value; _conversationId = json["conversationId"].Value; var cardName = json["cardName"].Value; await SendCardAsync(connectorClient, _conversationId, user, bot, cardName); return new ContentResult() { Content = "Notification received and send out: " + content.ToString(), ContentType = "text/html", StatusCode = (int)HttpStatusCode.OK, }; } private static Attachment CreateAdaptiveCardAttachment(string cardName){ var cardResourcePath = Path.Combine(".", "Resources", cardName); var adaptiveCardJson = System.IO.File.ReadAllText(cardResourcePath, Encoding.UTF8); var adaptiveCardAttachment = new Attachment() { ContentType = "application/vnd.microsoft.card.adaptive", Content = JsonConvert.DeserializeObject(adaptiveCardJson), }; return adaptiveCardAttachment; } private static async Task SendCardAsync(ConnectorClient connectorClient, string conversationId, string recipientId, string fromId, string cardName) { Activity messageActivity = new Activity(); messageActivity.Type = ActivityTypes.Message; var attachments = new List(); Activity reply = new Activity(); messageActivity.Attachments = attachments; messageActivity.Attachments.Add(CreateAdaptiveCardAttachment(cardName)); messageActivity.ChannelId = Channels.Msteams; messageActivity.ServiceUrl = "https://smba.trafficmanager.net/emea/"; messageActivity.Conversation = new ConversationAccount() { Id = conversationId }; messageActivity.Recipient = new ChannelAccount() { Id = recipientId }; messageActivity.From = new ChannelAccount() { Id = fromId }; await connectorClient.Conversations.SendToConversationAsync(messageActivity); } } } ``` ### Adaptive Cards As you can see in the json payload we are handing over a property called _cardName_, so we need to make sure to add this card to the bot project. The card will be added to the _Resources_ folder of the bot project. The card will be called _oppCard.json_ and will look like this: ```json { "type": "AdaptiveCard", "$schema": "http://adaptivecards.io/schemas/adaptive-card.json", "version": "1.5", "body": [ { "type": "Container", "style": "attention", "bleed": true, "items": [ { "type": "TextBlock", "text": "REMINDER!", "wrap": true, "style": "heading", "weight": "Bolder" } ] }, { "type": "Container", "bleed": true, "spacing": "None", "items": [ { "type": "Image", "url": "https://someImage.png", "spacing": "None", "horizontalAlignment": "Center" }, { "type": "Container", "items": [ { "type": "TextBlock", "text": "Contoso Chatbot Project is soon due!", "wrap": true, "size": "Large", "weight": "Bolder" }, { "text": "Status: Open", "type": "TextBlock", "weight": "Bolder" }, { "text": "Est. Close Date: 2021-12-31", "type": "TextBlock" } ], "style": "emphasis", "spacing": "None", "bleed": true } ] } ], "actions": [ { "type": "Action.Submit", "title": "Show me all Contoso opps", "data": { "msteams": { "type": "messageBack", "displayText": "Show me all Contoso opps", "text": "Show me all Contoso opps" } } } ] } ``` If everything is in place you can use Postman or Curl to send the notification to the bot. The notification will look like this (remember to insert the correct values for botId, userId and conversationId): ```json { "botId": "28:botid", "userId": "29:userid", "conversationId": "19:conversationid", "cardName": "oppCard.json" } ``` If everything is ok, then your bot should send out a new proactive message to the user which you set in the userId property. The message should look like this: {{< figure src="/images/070_message_demo.png" >}} ## Conclusion In this blog post we have seen how to create a proactive bot in Microsoft Teams. We have also seen how to create an Adaptive Card and how to send it out to the user. If you have any questions or comments, please let me know in the comments below. --- title: "Test ChatGPT for tutorial writing" url: https://bisser.io/test-chatgpt-for-tutorial-writing/ markdown: https://bisser.io/test-chatgpt-for-tutorial-writing/index.md date: 2023-03-14 lastmod: 2023-03-14 tags: AI, Bot Framework, Teams Development categories: AI, Bot Framework description: "Read this post to see how to use ChatGPT for supporting you in writing tech tutorials" status: current reading_time_min: 4 --- # Introduction As ChatGPT is currently everywhere, I wanted to test it out and see how it works. I also wanted to see if it can be used to write tutorials. So I decided to write this tutorial using ChatGPT. I will explain how I did it and what I think about it. So what I basically wanted to see is if it can write me a tutorial which lists all the steps I need to do to create a Teams bot. I also wanted to see if it can write the code for me. So I decided to use [Yo Teams](https://github.com/pnp/generator-teams) as a starting point. Yo Teams is a Yeoman generator for creating Microsoft Teams apps. It can be used to create a new Teams app project with all the necessary files and configurations. It can also be used to create a new bot project with all the necessary files and configurations. So I thought it would be a good starting point for my tutorial. ## The input The instruction I gave to ChatGPT was the following: _write a step by step tutorial on building a chatbot using yo teams_ And this was the outcome: ## The result 1. Install Node.js and npm on your computer if you don't have them already. Yo Teams is built on top of Yeoman, a scaffolding tool for web development projects that runs on Node.js. 2. Install Yo Teams globally by running the following command in your terminal: ``` npm install -g yo @microsoft/generator-teams ``` This will install Yo Teams and its dependencies on your system. 3. Create a new directory for your project and navigate to it in your terminal. 4. Run the Yo Teams generator by typing the following command: ``` yo teams ``` This will launch the Yo Teams wizard, which will guide you through the process of creating a new Teams project. 5. Select the "Bot" option from the wizard menu. This will create a new bot project template with all the necessary files and configurations. 6. Follow the prompts to enter your bot's display name, description, and other details. 7. Choose the language you want to use for your bot. Yo Teams supports several popular programming languages, including TypeScript, JavaScript, and C#. 8. Choose the messaging extension option if you want to create a bot that can be used as a messaging extension within Teams. This allows users to interact with your bot in the context of a chat conversation. 9. Choose the authentication option that best suits your needs. Yo Teams supports several authentication providers, including Azure Active Directory and Bot Framework Authentication. 10. Once you have completed the prompts, Yo Teams will generate a new bot project for you with all the necessary files and configurations. 11. Open the project in your preferred code editor and navigate to the "bot.ts" file. This is where you will write the code for your bot. 12. Write your bot's logic using the Bot Framework SDK. The Bot Framework SDK provides a set of libraries and tools for building bots that can interact with users through text messages, voice commands, and other channels. 13. Test your bot locally by running the following command in your terminal: ``` npm start ``` This will start your bot on your local machine, allowing you to test its functionality in a Teams conversation. 14. Once you are satisfied with your bot's functionality, deploy it to the Teams platform using the following command: ``` npm run build ``` This will build your bot project and create a package that can be uploaded to the Teams App Store. 15. Upload your bot package to the Teams App Store and make it available to your users. That's it! With these steps, you should now have a fully functional chatbot that can interact with users in Teams. ## The conclusion As you can see it got me a pretty solid tutorial even on the first attempt for a tool which is not that famous as others might be. The only real error I could detect was line number 8 where it should speak about the bot part as we are building a bot and not a messaging extension. But I think it is a pretty good result for a first attempt. I think it can be used to write tutorials and it can be used to write code. I think it is a good tool to have in your toolbox. I will definitely use it in the future. --- title: "Bot Framework - Power Virtual Agents comparison" url: https://bisser.io/bot-framework-power-virtual-agents-comparison/ markdown: https://bisser.io/bot-framework-power-virtual-agents-comparison/index.md date: 2022-09-12 lastmod: 2022-09-12 tags: AI, Bot Framework, Teams Development categories: AI, Bot Framework description: "This post covers the comparison between Bot Framework and Power Virtual Agents" status: legacy reading_time_min: 3 --- In recent years, there has been a growing interest in chatbots and virtual agents as businesses seek to improve their customer service and automate their workflows. Microsoft, one of the leading tech companies in the world, offers two popular options for building chatbots and virtual agents: the Microsoft Bot Framework and Power Virtual Agents. In this article, we'll compare and contrast the two options to help you decide which one is right for your needs. ## Overview of the Microsoft Bot Framework and Power Virtual Agents The Microsoft Bot Framework is a platform that allows developers to create intelligent bots for a wide range of platforms, including web, mobile, and even voice assistants like Amazon Alexa and Google Assistant. The Bot Framework provides developers with a set of tools and SDKs that enable them to create bots that can understand natural language and carry out complex conversations with users. Power Virtual Agents, on the other hand, is a no-code platform that allows business users to build chatbots and virtual agents without any programming knowledge. Power Virtual Agents offers a drag-and-drop interface that allows users to create and deploy chatbots quickly and easily. ## Ease of Use One of the key differences between the Bot Framework and Power Virtual Agents is their ease of use. While the Bot Framework provides developers with a robust set of tools for building chatbots, it requires programming skills and technical knowledge to use effectively. Developers need to have experience with programming languages like C# or JavaScript, as well as knowledge of machine learning and natural language processing. In contrast, Power Virtual Agents is designed to be used by business users with no coding experience. The platform provides a simple drag-and-drop interface that allows users to create chatbots and virtual agents using pre-built templates and workflows. This makes it much easier for non-technical users to build and deploy chatbots. ## Functionality When it comes to functionality, the Bot Framework offers more flexibility and customization options than Power Virtual Agents. With the Bot Framework, developers can build custom chatbots that can handle complex conversations and integrate with other services and platforms. The Bot Framework also provides access to Azure Cognitive Services, which includes powerful tools for natural language processing, speech recognition, and sentiment analysis. Power Virtual Agents, on the other hand, is designed for simpler use cases and workflows. The platform provides pre-built templates and workflows that cover common use cases like customer service, IT support, and HR inquiries. While Power Virtual Agents can be customized to some extent, it doesn't offer the same level of flexibility as the Bot Framework. ## Pricing Both the Bot Framework and Power Virtual Agents offer a range of pricing options, depending on your needs. The Bot Framework is available for free, but users need to pay for any Azure services they use, such as Azure Cognitive Services or Azure Bot Service. Pricing for these services can vary depending on usage. Power Virtual Agents is also available for free, but users need to have a Microsoft 365 license to use the platform. This means that users will need to pay for a Microsoft 365 subscription, which can be expensive for larger organizations. ## Conclusion In conclusion, both the Microsoft Bot Framework and Power Virtual Agents have their strengths and weaknesses, depending on your needs. If you're a developer with technical skills and need to build complex, customized chatbots, the Bot Framework is the better option. However, if you're a business user without coding skills and need to build simple chatbots quickly, Power Virtual Agents is a great choice. Ultimately, the decision between the two platforms will depend on your specific needs and resources. --- title: "Bot Framework Composer Series - 4 - Send a user profile card" url: https://bisser.io/bot-framework-composer-series-4-send-a-user-profile-card/ markdown: https://bisser.io/bot-framework-composer-series-4-send-a-user-profile-card/index.md date: 2022-03-25 lastmod: 2022-03-25 tags: AI, Bot Framework, Teams Development categories: AI, Bot Framework description: "This post covers how to create a user profile card using Adaptive Cards and Bot Framwork Composer" status: legacy reading_time_min: 3 --- As I was building some demo bots quite recently, I was also dealing a lot with including user profile information in a bot using the [Microsoft Graph](https://developer.microsoft.com/en-us/graph) and [Bot Framework Composer](https://docs.microsoft.com/en-us/composer/)! ## Build the dialog The first thing **you** need to do after creating a new bot in Composer (doesn't matter which template you choose), is to add a new trigger for the dialog. In this tutorial the dialog is called GetMyUserProfileDialog and will be used to get the necessary information from the Graph and display it using an Adaptive Card: {{< figure src="/images/067-dialog1.png" >}} The first action we need in this dialog is to make sure that the user is authenticated. Therefore, we will begin a new dialog called **OauthDialog** which will make sure to authenticate the user if the user is not already logged in: {{< figure src="/images/067-dialog2.png" >}} Within the OauthDialog, we simply use the action **OAuth login** which takes the connection name of the OAuth Connection Setting, which you need to configure on your Azure Bot (follow [this tutorial](https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-authentication?view=azure-bot-service-4.0) to configure everything): {{< figure src="/images/067-dialog3.png" >}} ## Install and use the Microsoft Graph package Now that the authentication flow is set up, we can install a package from the package manager in Composer to ease the use of the Microsoft Graph. Simply install the **Microsoft.Bot.Components.Graph** package as shown below: {{< figure src="/images/067-package1.png" >}} This will then add more actions to your list of available actions in Composer. Now we can use the action **"Microsoft Graph Get Signed-in User's Profile"** which only takes the _token_ as a parameter and you do not need to take care of the rest to connect to the Graph: {{< figure src="/images/067-package2.png" >}} The second thing we need for our profile card is the user's profile picture. Again we take advantage of the Microsoft Graph package to retreive that, by using the **"Microsoft Graph - Get Profile Photo"** which takes the _token_ and the _user ID_ as parameters and returns the user profile picture to be used later on: {{< figure src="/images/067-package3.png" >}} ## Create the Adaptive Card template Now that we have all information we need, we can go ahead and create the Adaptive Card using the [Adaptive Card Designer](https://adaptivecards.io/designer): ```json { "type": "AdaptiveCard", "$schema": "http://adaptivecards.io/schemas/adaptive-card.json", "version": "1.3", "body": [ { "type": "TextBlock", "text": "${user.profile.displayName}", "wrap": true, "size": "Medium", "weight": "Bolder" }, { "type": "Container", "items": [ { "type": "ColumnSet", "columns": [ { "type": "Column", "width": "auto", "items": [ { "type": "Image", "url": "data:image/png;base64,${user.photo}", "size": "Large", "horizontalAlignment": "Center" } ], "horizontalAlignment": "Center" }, { "type": "Column", "width": "stretch", "items": [ { "type": "TextBlock", "text": "Given name: ${user.profile.givenName}", "wrap": true }, { "type": "TextBlock", "text": "Surname: ${user.profile.surname}", "wrap": true }, { "type": "TextBlock", "text": "Mail: ${user.profile.mail}", "wrap": true }, { "type": "TextBlock", "text": "Jobtitle: ${user.profile.jobTitle}", "wrap": true }, { "type": "TextBlock", "text": "Office: ${user.profile.officeLocation}", "wrap": true } ] } ] } ] } ], "msteams": { "width": "Full" } } ``` After the card design is ready, we can simply copy the JSON payload into the value of a new property called _turn.profileCard_ > IMPORTANT: make sure to select {} object as a type {{< figure src="/images/067-card1.png" >}} Now the last thing we need is to send out the Adaptive Card using the _Send a response_ action with the card payload attached as an attachment: {{< figure src="/images/067-card2.png" >}} ## Test the bot After you have started or deployed your bot (depending if you're running a tunneling software on your machine or not), you should be able to test your bot in Microsoft Teams and after the bot will ask you to authenticate, you should see your profile details in a nice Adaptive Card: {{< figure src="/images/067-test1.png" >}} ## Summary It is quite easy to leverage the Microsoft Graph using Bot Framework Composer by simply adding the Microsoft Graph package to it. . If you haven't check out the other parts of my Bot Framework Composer series [Use Adaptive Cards ](https://bisser.io/bot-framework-composer-series-1-use-adaptive-cards/), [Property management](https://bisser.io/bot-framework-composer-series-2-property-management/) & [Teams Task Modules](https://www.bisser.io/bot-framework-composer-series-3-teams-task-modules/) --- title: "Bot Framework Composer Series - 3 - Teams Task Modules" url: https://bisser.io/bot-framework-composer-series-3-teams-task-modules/ markdown: https://bisser.io/bot-framework-composer-series-3-teams-task-modules/index.md date: 2021-07-15 lastmod: 2021-07-15 tags: AI, Bot Framework categories: AI, Bot Framework description: "This post covers how to create a Microsoft Teams Task Module using Bot Framework Composer" status: legacy reading_time_min: 5 --- While there is the option to create a [Microsoft Teams Task Module](https://docs.microsoft.com/en-us/microsoftteams/platform/task-modules-and-cards/what-are-task-modules) using code a already a while ago, you would exactly need to know how to properly use the various methods and actions in your bot project. Therefore, it's now possible to build a task module using [Bot Framework Composer](https://docs.microsoft.com/en-us/composer/) without writing (almost) any code! As I also had the opportunity to demo the things covered in this post on the PnP call earlier this year, you can also follow along by watching the tutorial here: {{< youtube mgoCHlY0AX8 >}} ## Installing the Teams package in Composer The first thing you need to do after creating a new bot in Composer (doesn't matter which template you choose), is to add the Microsoft.Bot.Components.Teams package using the package manager: {{< figure src="/images/066-package-manager.png" >}} Now that we have this package installed, you can see that another entry on the "Add action" menu appears called "Microsof Teams" which offers a variety of options to either work with messaging extensions, messages or task modules: {{< figure src="/images/066-teams-actions.png" >}} ## Trigger the task module with a card button The next bit we need is something which should trigger our task module. In this case we'll use a simple hero card which contains a button to trigger the task module. The hero card for this example looks like this and is part of the bot's greeting trigger: ```md [HeroCard subTitle = If you want to order a pizza just click here: buttons = ${[orderTaskModuleFetchButton()]} ] ``` As you might notice, this hero card refers to another function called `orderTaskModuleFetchButton` which we will need to define in the bot respones before we can continue. Therefore, we will need to add the following snippet to the bot responses common section: ```md # cardActionTemplate(title, text, type, value) [CardAction Type = ${type} Title = ${title} Text = ${text} Value = ${value} ] # orderTaskModuleFetchButton() - ${cardActionTemplate('Order Pizza', 'OrderFetch', 'invoke', json({type: "task/fetch", buttonId: "OrderFetch"}))} ``` ## Creating a new Teams-specific trigger So when a user clicks that button in the hero card, the bot should execute the onTaskModuleFetch action to present the user with a task module. This can be achieved by adding a new trigger to the dialog of type Microsoft Teams and with the event "On task module fetch": {{< figure src="/images/066-add-trigger.png" >}} To make sure that this trigger is only triggered when a user clicks the button defined in the hero card above, we need to add a condition to this trigger to check if the id of the button clicked is the id defined in our hero card: ```md =turn.activity.value.data.buttonId == "OrderFetch" ``` The in the trigger "On task module fetch" we need to add a new action of type "Send task module card response" to actually send a new Adaptive Card within a task module after clicking the button. Add a new attachment of type Adaptive Card with the following content to that action: ```json { "$schema": "http://adaptivecards.io/schemas/adaptive-card.json", "type": "AdaptiveCard", "version": "1.3", "body": [ { "type": "Container", "items": [ { "type": "Container", "items": [ { "type": "ColumnSet", "columns": [ { "type": "Column", "width": "stretch", "bleed": true, "items": [ { "type": "TextBlock", "text": "Just select the type of pizza to order:", "weight": "Bolder", "size": "Medium", "wrap": true } ] } ], "bleed": true } ], "bleed": true } ], "bleed": true }, { "type": "Input.ChoiceSet", "choices": [ { "title": "Diavolo", "value": "Diavolo" }, { "title": "Hawaii", "value": "Hawaii" }, { "title": "Prosciutto", "value": "Prosciutto" }, { "title": "Margherita", "value": "Margherita" }, { "title": "Funghi", "value": "Funghi" }, { "title": "Al tonno", "value": "Al tonno" } ], "placeholder": "Select your favorite pizza", "id": "pizzaType" } ], "actions": [ { "type": "Action.Submit", "title": "Order" } ] } ``` And don't forget to give your action a title and set height and width as needed. Now when running the bot and starting a conversation in Teams, you should get the hero card as part of the greeting and when you click on the button, the bot should already send a new task module (remember starting NGROK beforehand): {{< figure src="/images/066-task-module.png" >}} ## Handle the task module submit action The last missing piece is to handle the onTaskModuleSubmit action which gets executed whenever a user finishes the task module process (e.g.: in our demo it is ordering a pizza). Therefore we need to add another trigger of type "On task module submit" which can be found in the Microsoft Teams triggers: {{< figure src="/images/066-submit-trigger.png" >}} Within that trigger we add a new action "Send task module message respose" which will allow us to add a message to the already open task module. Within this message we can also refer to the data or information entered by the user in the task module using `${turn.activity.value.data.yourProperty}` like in this case we can refer to the pizza type chosen by the user in the task module: {{< figure src="/images/066-submit-response.png" >}} After restarting the bot and switching back to Teams, you can now again trigger the execution of the task module and pick a pizza for ordering. When you click on order after selecting a pizza, you should see the message "Thank you for ordering a pizza [type]" as expected: {{< figure src="/images/066-demo.gif" >}} ## Summary As you can see, it is quite easy to build task modules for Teams using Bot Framework Composer. There is no real need to do this using code anymore, as you can do pretty much everything right from Composer. If you haven't check out the other parts of my Bot Framework Composer series [Use Adaptive Cards ](https://bisser.io/bot-framework-composer-series-1-use-adaptive-cards/) & [Property management](https://bisser.io/bot-framework-composer-series-2-property-management/) --- title: "Conversational AI updates from MSBuild 2021" url: https://bisser.io/conversational-ai-updates-from-msbuild-2021/ markdown: https://bisser.io/conversational-ai-updates-from-msbuild-2021/index.md date: 2021-05-26 lastmod: 2021-05-26 tags: AI, Bot Framework categories: AI, Bot Framework description: "This post covers all the great announcements from this years Microsoft Build conference 2021" status: legacy reading_time_min: 2 --- ## Microsoft Bot Framework In this section you'll find the list of all updates related to the Microsoft Bot Framework (check out the official blog post on all the updates from [here](https://aka.ms/ConvAIBuild2021)): ### Bot Framework Composer {{< figure src="/images/065-composer-intro.png" >}} - Bot Framework Composer 2.0 is GA - Remember, in [my blog post about Microsoft Build updates last year](https://bisser.io/conversational-ai-updates-from-msbuild-2020/) I reported, that Composer has been gone GA . Now 1 year forward, we now get the next major version of Composer with lots of great feautures and updates - You can either download the desktop app from [here for Windows](https://aka.ms/bf-composer-download-win), [here for MacOS](https://aka.ms/bf-composer-download-mac) or [here for Linux](https://aka.ms/bf-composer-download-linux) - Bot Framework Composer 2.0 introduces new templates (both in C# & Node (Preview)): - Empty Bot - Core Bot with Language - Core Bot with QnA Maker - Core Assistant Bot - Enterprise Assistant Bot - Enterprise Calendar Bot - Enterprise People Bot - [Bot Framework Components](https://docs.microsoft.com/en-us/composer/how-to-manage-package) are now available as a package management solution in Composer: - You can now install reusable components built by [Microsoft](https://github.com/microsoft/botframework-components) or the [Bot Framework Community](https://github.com/BotBuilderCommunity) or you can even [build your own reusable packages](https://docs.microsoft.com/en-us/composer/how-to-create-packages) for your Bot Framework Composer based bots - Publishing to Azure has been made easier - The [publishing process](https://docs.microsoft.com/en-us/composer/how-to-provision-azure-resources) of your Composer built bots is getting a lot easier with the release of the integrated resource provisioning and publishing: - {{< figure src="/images/065-composer-azure-publishing.png" >}} - New interaction types like speech and cards available in Composer to ease the authoring of text & voice bots by having a rich authoring experience including [SSML](https://docs.microsoft.com/en-us/azure/cognitive-services/speech-service/speech-synthesis-markup?tabs=csharp): - {{< figure src="/images/065-composer-interaction-types.png" >}} - New Adaptive Runtime for all Bot Framework bots (Composer & SDK) which encapsulates many capabilities like multilingual support or access to Cognitive Services available - [Power Virtual Agent integration in Bot Framework Composer](https://powervirtualagents.microsoft.com/en-us/blog/power-virtual-agents-integration-with-bot-framework-composer-is-now-generally-available-2/) is now GA - Consuming and developing skills in Composer has been improved and simplified ### Conversational AI Platform - [Orchestrator](https://docs.microsoft.com/en-us/composer/concept-orchestrator) deplaces Bot Framework Dispatch as a dispatching solution - [Conversational User Experience Guide](https://docs.microsoft.com/en-us/composer/design/conversational-user-experience-design) has been released with lots of useful information on how to build a good CUX - {{< figure src="/images/065-cux.png" >}} ### Azure Bot Service - Azure Bot Service [Telephony Channel](https://docs.microsoft.com/en-us/azure/bot-service/bot-service-channel-connect-telephony?view=azure-bot-service-4.0) in preview - Azure Monitor support in Azure Bot Service - Customer Managed Encryption Keys available in Azure Bot Service ## Azure Cognitive Services ### Translation - [Translator announces general availability of Document Translation](https://techcommunity.microsoft.com/t5/azure-ai/translator-announces-general-availability-of-document/ba-p/2382171) ## Conversational AI sessions from Build - [Build intelligent applications infused with world-class AI](https://mybuild.microsoft.com/sessions/2ba55238-d398-46f9-9ff2-eafcd9d69df3?source=sessions) - [Ask the Experts: Build intelligent applications infused with world-class AI](https://mybuild.microsoft.com/sessions/edc5aef9-dec8-40d4-b704-028117f6350b?source=sessions) - [Extend your Power Virtual Agents with Bot Framework Composer](https://mybuild.microsoft.com/sessions/13222b6b-0429-4cfd-ba76-4ed06e8a9f05?source=sessions) --- title: "Bot Framework Composer Series - 2 - Property management" url: https://bisser.io/bot-framework-composer-series-2-property-management/ markdown: https://bisser.io/bot-framework-composer-series-2-property-management/index.md date: 2020-08-23 lastmod: 2020-08-23 tags: AI, Bot Framework categories: AI, Bot Framework description: "While building chatbots using Bot Framework Composer I often encoutered the problem that I needed to store properties and use them within my dialogs for different purposes. Within the Bot Framework SDK, I would most probably use the state to store information which is not only valuable within a s..." status: legacy reading_time_min: 4 --- This is the second part of my Bot Framework Composer blog series. If you want to know more about Adaptive Cards in Composer, take a look at the [first part](https://bisser.io/bot-framework-composer-series-1-use-adaptive-cards/) ## Introduction While building chatbots using [Bot Framework Composer](https://github.com/microsoft/BotFramework-Composer) I often encoutered the problem that I needed to store properties and use them within my dialogs for different purposes. Within the Bot Framework SDK, I would most probably use the [state](https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-concept-state?view=azure-bot-service-4.0) to store information which is not only valuable within a single dialog or turn, but for the whole conversation. The equivalent of the state in Composer is called "memory" which can be described as the bot's mind or brain. But unlike within the SDK, the concept of storing properties is a bit different from the state approach, where you basically have user and conversation state. Therefore, this post will give you an overview on how to manage your properties accordingly within your Composer based chatbot. ## Property scopes Within Composer there are four different property scopes: - user - conversation - dialog - turn In Composer, property names are always in the format "scope.name" (e.g.: user.firstName). Therefore, the scope of the property is actually part of the property's name. When setting a new property you'll need to make sure that you choose the correct scope, otherwise you might end up using a property which can't be used in the current context. This can happen if you create a property "turn.choiceResult" in one dialog which you want to access in another dialog. As the scope of this property is turn, the value can't be accessed outside of this particular turn. Therefore you would need to use another scope for your property to fulfill your use case. All of these scopes have a different property lifetime, which are described below... ### user The user scope is tied to the user. This is why properties in the user scope are stored indefinetly for a specific user. Use this scope if you want to store user information such as the following within your bot: - General information such as name, birthday, age, ... - Location information such as address, street, postal code, country, ... - Contact informaiton such as email address, telephone number, ... To create a new property using the **user scope**, simply name your property `user.propertyName` (e.g.: user.name or user.age). ### conversation In contrast to the user scope, the conversation scope is tied to one specific conversation (identified through the conversationID). Like the user scope, the properties in the conversation scope are kept forever, however they could be accessed by multiple users within the same conversation. Examples use cases for using the conversation scope may be: - General information about the conversation (e.g.: conversation language) - Current topic of the conversation - Wether the bot already asked a specific question within the conversation - Poll responses given by users within a specific conversation If you want to create a new conversation-scoped property just use `conversation.propertyName` as the property's name (e.g.: conversation.topic). > **NOTE:** The conversation scope is not always only bound to one specific user only. For instance, when your bot is used in a Microsoft Teams team or group conversation, multiple users are part of the same conversation and therefore share the same conversationID. So please use this scope accordingly. ### dialog The dialog scope, as the name already suggests, is bound to the active dialog and all parent or child dialogs of the active one. Therefore, the properties in that scope are accessible until the last active dialog in that dialog stack ends (look at [this doc](https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-concept-dialog?view=azure-bot-service-4.0) for more information on how dialogs work). The following examples make perfect sense, when used in the dialog scope: - Answers to prompts within a given dialog which are used in further turns ### turn The turn scope is bound to one specific turn within the conversation. A turn is basically described as handling one specific message from the user, therefore this scope is the one with the lowest lifetime as these kind properties are only retained until the bot has successfully handled the user's message. Thus, turn-scoped properties are often used to cache certain values before persisting them into other scopes, like dialog or user. Three out of the box property within **turn scope**, however, are: - turn.activity.text holds the user message within the current turn - turn.activity.type holds the activity type of the current turn (e.g.: message, conversationUpdate, typing, endOfConversation, messageReaction, ...) - turn.activity.locale holds the locale of the locale of the turn.activity.text property (e.g.: en-us, de-de, ...) ## Summary Properties are an essential part within Bot Framework Composer. They are an easy way to share information across the whole conversation and therefore minimize the effort to implement a bot. However, you should always pick the scope accordingly to prevent, that properties are shared between users if they shouldn't or even worse, properties are in a scope which do not provide the needed property lifetime (e.g. turn scope) 🤖 --- title: "Visual Studio Code extensions for Bot Framework devs" url: https://bisser.io/visual-studio-code-extensions-for-bot-framework-devs/ markdown: https://bisser.io/visual-studio-code-extensions-for-bot-framework-devs/index.md date: 2020-07-27 lastmod: 2020-07-27 tags: AI, Bot Framework categories: AI, Bot Framework description: "As a Bot Framework enthusiast, I was looking for ways how to speed up routines in my daily tasks. Therefore I decided to publish some nice little helpers together with my friends from the Bot Framework Community to speed up tasks when working on a bot a bit while working with VS Code." status: legacy reading_time_min: 2 --- ## Motivation As a Bot Framework enthusiast, I was looking for ways how to speed up routines in my daily tasks. Therefore I decided to publish some nice little helpers together with my friends from the [Bot Framework Community](https://github.com/BotBuilderCommunity) to speed up tasks when working on a bot a bit. And this is what we currently published: **All Visual Studio Code snippets can be downloaded from [here](http://bit.ly/bfcvscode).** **All Visual Studio Code commands can be downloaded from [here](http://bit.ly/bfcvscodecommands).** ## Snippets Currently we have snippets available for the following categories: - Bot Framework Web Chat - LUIS (.lu) files - Adaptive Cards ### Create a basic Web Chat HTML file structure - bf-wc-create-basic-hml The snippet `bf-wc-create-basic-hml` creates a basic plain html/js web chat within an HTML file: {{< figure src="https://raw.githubusercontent.com/BotBuilderCommunity/botbuilder-community-tools/master/bfc-vscode-snippets/assets/bfc-webchat-basic-snippet-demo.gif" >}} ### Create a Web Chat HTML file structure with advanced styling - bf-wc-create-style-html The snippet `bf-wc-create-style-html` creates a plain html/js web chat within an HTML file including some styling options already added to the web chat: {{< figure src="https://raw.githubusercontent.com/BotBuilderCommunity/botbuilder-community-tools/master/bfc-vscode-snippets/assets/bfc-webchat-styling-snippet-demo.gif" >}} ### Create a sample Language Understanding (.lu) file - bfc-lu-create The snippet `bfc-lu-create` creates a the required structure within .lu files used for managing LUIS applications: {{< figure src="https://raw.githubusercontent.com/BotBuilderCommunity/botbuilder-community-tools/master/bfc-vscode-snippets/assets/bfc-lu-snippet-demo.gif" >}} ### Create a sample Adaptive Card "Activity Update" - bf-ac-activityUpdate Adds a [activity update sample card](https://adaptivecards.io/samples/ActivityUpdate.html) to your project. ### Create a sample Adaptive Card "Input Form" - bf-ac-inputForm Initializes a new [input form sample card](https://adaptivecards.io/samples/InputForm.html) to be used to gather input from users. ### Create a sample Adaptive Card "Weather Compact" - bf-ac-weatherCompact Creates the [weather compact sample card](https://adaptivecards.io/samples/WeatherCompact.html) to demo a weather card. ### Create a sample Adaptive Card "Expense Report" - bf-ac-expenseReport Provides you with the [expense report sample card](https://adaptivecards.io/samples/ExpenseReport.html) to include in your bot's project. ## Commands At the moment we published one command which lets you import a new LUIS app usign the [Bot Framework CLI](https://github.com/microsoft/botframework-cli) from a .lu file (which you could create with our snippets) to make the LUIS deployment process far easier. It can be done like this: {{< figure src="https://raw.githubusercontent.com/BotBuilderCommunity/botbuilder-community-tools/master/bfc-vscode-commands/assets/importLuisApp.gif" >}} All you need to do is to create a .lu file and populate it and then right-click it from the explorer view and select **Import LUIS application** fill out the details and the command will create a new LUIS application for you! > If you have any ideas or feeback, please feel free to share it [here](https://github.com/BotBuilderCommunity/botbuilder-community-tools) or do a PR 😎 --- title: "Conversational AI updates from MSBuild 2020" url: https://bisser.io/conversational-ai-updates-from-msbuild-2020/ markdown: https://bisser.io/conversational-ai-updates-from-msbuild-2020/index.md date: 2020-05-20 lastmod: 2020-05-20 tags: AI, Bot Framework categories: AI, Bot Framework description: "This post covers all the great announcements from this years Microsoft Build conference 2020" status: legacy reading_time_min: 3 --- ## Microsoft Bot Framework In this section you'll find the list of all updates related to the Microsoft Bot Framework (check out the official blog post on all the updates from [here](https://techcommunity.microsoft.com/t5/azure-ai/build-2020-conversational-ai-updates/ba-p/1397685)): ### Bot Framework Composer {{< figure src="/images/062-composer-intro.png" >}} - Bot Framework Composer is GA - You can either download the desktop app from [here for Windows](https://aka.ms/bf-composer-download-win), [here for MacOS](https://aka.ms/bf-composer-download-mac) or [here for Linux](https://aka.ms/bf-composer-download-linux) - If you want to run Composer locally as a web-based component, you can download and build Composer from Github following [this link](https://github.com/microsoft/BotFramework-Composer) - Bot Framework Composer Docs are [here](https://aka.ms/bfcomposer) - Bot Framework Composer allows to build and export [Bot Framework Skills](https://docs.microsoft.com/en-us/azure/bot-service/skills-conceptual?view=azure-bot-service-4.0) or you can also export the Bot Framework Composer assets (lg, lu, settings and dialog files) which you can then plug into other bots: {{< figure src="/images/062-composer_export.png" >}} ### Bot Framework SDK - Bot Framework SDK v4.9 released - Release notes can be found [here](https://github.com/microsoft/botframework-sdk/issues/5836) with a ton of updates: - [Adaptive Dialogs](https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-adaptive-dialog-introduction?view=azure-bot-service-4.0) are GA - [Skills](https://docs.microsoft.com/en-us/azure/bot-service/skills-conceptual?view=azure-bot-service-4.0) support Adaptive Dialogs and have improved support for SSO and OAuth - Improvements around Microsoft Teams specific funtionality including previews for the Java SDK as well as the developer preview of SSO capabilities in Microsoft Teams - Bot Telemtry improvments including the ability to map dialogs into Azure Application Insights Page View Events - Health Check APIs available to see if a bot is still running - CLI tools for Adaptive Dialogs available [here](https://github.com/Microsoft/botframework-cli) - [Language Generation](https://docs.microsoft.com/en-us/composer/concept-language-generation) is now GA in C# and JS SDKs - [Adaptive Expressions](https://aka.ms/adaptive-expressions) is now GA in C# and JS SDKs - Authentication improvments for bots and skills are now available to be used like support for certificate based authentication - [Generated Dialogs](https://github.com/microsoft/BotBuilder-Samples/tree/master/experimental/generation) is in early preview stage - [Bot Framework Adaptive Tool](Bot Framework Adaptive Tool) is in preview ### Azure Bot Service - [Direct Line App Service Extension](https://aka.ms/bf-directline-ase) is now GA - [Human Handoff capabilities](https://docs.microsoft.com/en-us/azure/bot-service/bot-service-design-pattern-handoff-human?view=azure-bot-service-4.0) have been improved - [Alexa channel](https://docs.microsoft.com/en-us/azure/bot-service/bot-service-channel-connect-alexa?view=azure-bot-service-4.0) has been added natively into the Azure Bot Service (Kudos to [Gary Pretty](https://twitter.com/GaryPretty) for doing that👍) ### Virtual Assistant - [Virtual Assistant](http://aka.ms/VirtualAssistant) is now GA - Bot Framework Composer can now be used to build Bot Framework Skills to be used in virtual assistant {{< figure src="/images/062-va.png" >}} ## Azure Cognitive Services ### Language Understanding - New update to the [LUIS](https://luis.ai) portal with improvements to the labelling expirience - [Container support](https://docs.microsoft.com/en-us/azure/cognitive-services/luis/luis-container-howto?tabs=v3) for LUIS apps - Upgraded machine learned entities which are replacing composite and simple entities - [DevOps support](https://github.com/Azure-Samples/LUIS-DevOps-Sample) for language models - Bot Framework Orchestrator enters private preview {{< figure src="/images/062-luis.png" >}} Read the full blog of LUIS announcements [here](https://techcommunity.microsoft.com/t5/azure-ai/build-2020-language-understanding-s-new-portal-tools-and/ba-p/1401000). ### QnA Maker - New rich text editor support to streamline KB editing process - RBAC support for knowledge bases {{< figure src="/images/062-qnamaker.png" >}} ### Speech Services - Speech Services has now improved the accuracy of Speech to Text and Text to Speech - Custom commands and pronunciation assessment features have been added to create advanced speech capabilities ## Conversational AI sessions from Build - [Conversational AI powered Customer and Employee Virtual Assistants](https://mybuild.microsoft.com/sessions?q=INT139) - [Accelerate bot development in Power Virtual Agents](https://mybuild.microsoft.com/sessions?q=INT155) ## On demand content - [Bot Framework Composer: Bot Framework’s new collaborative Conversational AI development environment](https://youtu.be/r9WQPSaLnaU) - [Use the Efficiency of Low-Code with the Extensibility to Azure to Design World-Class Chatbots](https://youtu.be/oJWJA-U4-m8) - [Conversational AI and human agents working together](https://youtu.be/Z0IDiekbOp4) - [Author rich content in QnA Maker knowledge base and enable role based sharing](https://www.youtube.com/watch?v=lB9klrdrqOk&feature=youtu.be) --- title: "Bot Framework Teams Messaging Extensions Walkthrough" url: https://bisser.io/bot-framework-teams-messaging-extensions-walkthrough/ markdown: https://bisser.io/bot-framework-teams-messaging-extensions-walkthrough/index.md date: 2020-05-19 lastmod: 2020-05-19 tags: AI, Bot Framework categories: AI, Bot Framework description: "A step-by-step tutorial for building a Microsoft Teams messaging extension using the Microsoft Bot Framework to handle task management (To Do and Planer) directly within conversations!" status: legacy reading_time_min: 19 --- ## The Starting Point I was previously dealing quite extensively with the topic Bot Framework & Microsoft Teams, as Teams is becoming THE communication tool in the modern workplace. Myself, as well as we (the company I work for) have build quite a lot of bots, which are not targeting Teams exclusively but other channels like Web Chat or others as well. But I did a little project during the homeoffice time due to Covid-19 to finally build our very own bot which should serve me and my co-workers in our daily jobs. As we are quite dependent on Teams and do all of our calls, meetings and project work in Teams, I built a bot which is only targeting Teams at the moment. And this bot in it's first stage has 2 main use cases included: * QnA * Task management The QnA part is rather simple as it is utilizing LUIS & QnA Maker to answer questions stored in a knowledge base. But for the task management part, I wanted to do something quite new on the platform: [Teams Messaging Extensions](https://docs.microsoft.com/en-us/microsoftteams/platform/messaging-extensions/what-are-messaging-extensions). And those components are pretty useful in many use cases, may it be as a search based extension to grab stock images somewhere, or an action based extension to handle specific actions with it. And myself and many of my co-workers are dependent on a solid task management solution, I decided to "modernize" how can manage tasks right within a conversation. The awesome thing here is that with the solution shown in this post, we avoid the context-switch, meaning users do not need to go to another tool during a long running Teams conversation to note down a task they need to work on. All that happens now right from the conversation. So the following post describes how to setup your bot in a way to use a messaging extension to fulfill task management based on [Microsoft To Do](https://todo.microsoft.com/) and [Planner](https://www.microsoft.com/en-us/microsoft-365/business/task-management-software) (but of course you can take the code and exhange the task management tools to your preferred solution). > The code is published in the Bot Builder Community GitHub repo and can be found [here](https://github.com/BotBuilderCommunity/botbuilder-community-dotnet/tree/develop/samples/Teams%20Graph%20Messaging%20Extensions) (feel free to use or contribute) ## Create Azure Resources This post will only describe the process of having a local bot running and a Bot Channels Registration which we will use to connect our Teams bot to our local running bot using ngrok (More infos on how to setup ngrok for local bot development can be found [here](https://blog.botframework.com/2017/10/19/debug-channel-locally-using-ngrok/)). ### Azure AD App Registration The first thing we need is to setup a new Azure AD app registration, as we want to use the [Microsoft Graph](https://docs.microsoft.com/en-us/graph/overview) to handle the task management processing. Therefore we need to go over to our Azure portal and create a new Azure AD App registration (like shown [here](https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal#create-an-azure-active-directory-application)): {{< figure src="/images/061-01.png" >}} While creating your app registration you need to provied the url `https://token.botframework.com/.auth/web/redirect` as a redirect URI to establish a conncetion to the Bot Framework for grabbing your authentication token. Next up, we need to add some API permissions to our app to make sure we can use the Graph to perform certain tasks (don't forget to grant admin consent for those permissions after adding them): {{< figure src="/images/061-02.png" >}} ### Azure Bot Channels Registration After the Azure AD App Registration has been created, we can create a new Bot Channels Registration in the Azure portal: {{< figure src="/images/061-03.png" >}} After it has been created, we need to add our previously created app registration to the OAuth Connection Settings of our bot: {{< figure src="/images/061-04.png" >}} From here you need to provide the following details: {{< figure src="/images/061-05.png" >}} After saving you can validate your service provider connection setting to see if you can connect to your AAD app registration and get a token from there: {{< figure src="/images/061-06.png" >}} The last thing we need to grab is the Microsoft App ID and App secret for the Bot Channels Registration which can be found from the Azure AD App Registration pane as well. ## Create the Bot As I would only like to demo the Teams bits in this post, I will create everything in an echo bot template, but you can of course use it in our existing bots or go with any other BF template. The first thing we need to do after the bot has been created in VS or VS Code is to add our MicrosoftAppId and MicrosoftAppSecret in the appsettings.json of our bot as well as the , tenantId and the serviceUrl we will use from our bot: {{< figure src="/images/061-07.png" >}} Next up we need to add some NuGet packages to our bot which are used in this demo: * Microsoft.Bot.Builder * AdpativeCards * Microsoft.Graph * Microsoft.Graph.Beta So more or less you'll end up having the following NuGet packages added to your bot: {{< figure src="/images/061-08.png" >}} What we now need is to tweak our project file a bit, because we are using the Microsoft.Graph and the Microsoft.Graph.Beta NuGet packages, as the To Do implementation is still only available from the beta endpoints. So you would need to add the following lines to our csproj file to make both work at the same time: ```XML BetaLib ``` and: ```XML False Resources\Microsoft.Graph.Beta.dll BetaLib ``` ### Implement Graph Client After setting up the bot locally, we need to add the Microsoft Graph functionality to it. The easiest way of doing that is to create a new class called **SimpleGraphClient.cs** which contains the following code: ```C# extern alias BetaLib; using System; using System.Net.Http.Headers; using System.Threading.Tasks; using Beta = BetaLib.Microsoft.Graph; using Microsoft.Graph; namespace SBI_TeamsBot.Services { public class SimpleGraphClient { private readonly string _token; public SimpleGraphClient(string token) { if (string.IsNullOrWhiteSpace(token)) { throw new ArgumentNullException(nameof(token)); } _token = token; } public async Task GetTaskAsync() { var graphClient = GetAuthenticatedClient(); var tasks = await graphClient.Me.Outlook.Tasks.Request().GetAsync(); return tasks[0]; } public async Task GetFavoritePlans() { var graphClient = GetAuthenticatedClient(); var plans = await graphClient.Me.Planner.FavoritePlans.Request().GetAsync(); return plans; } public async Task GetCurrentPlan(string groupId) { var graphClient = GetAuthenticatedClient(); var plans = await graphClient.Groups[groupId].Planner.Plans.Request().GetAsync(); return plans; } public async Task CreatePlannerTaskAsync(string planId, string subject, string dueDate, string startTime, string userId) { var graphClient = GetAuthenticatedClient(); var assignments = new Beta.PlannerAssignments(); assignments.AddAssignee(userId); var plannerTask = new Beta.PlannerTask { DueDateTime = DateTimeOffset.Parse(dueDate), StartDateTime = DateTimeOffset.Parse(startTime), Title = subject, PlanId = planId, Assignments = assignments }; var plans = await graphClient.Planner.Tasks.Request().AddAsync(plannerTask); return plans; } public async Task CreateTaskAsync(string subject, string dueDate, string startTime, Beta.ItemBody body) { var graphClient = GetAuthenticatedClient(); var outlookTask = new Beta.OutlookTask { Subject = subject, StartDateTime = new Beta.DateTimeTimeZone { DateTime = startTime, TimeZone = "Eastern Standard Time" }, DueDateTime = new Beta.DateTimeTimeZone { DateTime = dueDate, TimeZone = "Eastern Standard Time" }, Body = body }; var res = await graphClient.Me.Outlook.Tasks.Request().Header("Prefer", "outlook.timezone=\"Pacific Standard Time\"").AddAsync(outlookTask); return res; } // Get an Authenticated Microsoft Graph client using the token issued to the user. private Beta.GraphServiceClient GetAuthenticatedClient() { var graphClient = new Beta.GraphServiceClient( new DelegateAuthenticationProvider( requestMessage => { // Append the access token to the request. requestMessage.Headers.Authorization = new AuthenticationHeaderValue("bearer", _token); // Get event times in the current time zone. requestMessage.Headers.Add("Prefer", "outlook.timezone=\"" + TimeZoneInfo.Local.Id + "\""); return Task.CompletedTask; })); return graphClient; } } } ``` This class will allow us to use the Microsoft Graph SDK to perform certain operations instead of using HTTP clients to perform those operations. ### Implement Adaptive Cards Helper As we want to provide a messaging extension which will dynamically fetch data using an Adaptive Card, we need to provide two additional classes to fulill that: * AdaptiveCardData.cs * This class will act as our Adaptive Card data model * AdaptiveCardHelper.cs * This class will help us generate the Adaptive Card based on the current context The AdaptiveCardData.cs will look as follows: ```C# using System; using System.Collections.Generic; using System.Linq; using System.Threading.Tasks; namespace SBI_TeamsBot.Models { public class AdaptiveCardData { public AdaptiveCardData() { MultiSelect = "false"; } public string SubmitLocation { get; set; } public string Question { get; set; } public string MultiSelect { get; set; } public string Option1 { get; set; } public string Option2 { get; set; } public string Option3 { get; set; } public string Option4 { get; set; } public string Option1Value { get; set; } public string Option2Value { get; set; } public string Option3Value { get; set; } public string Option4Value { get; set; } } } ``` The AdaptiveCardHelper.cs will look as follows: ```C# using System.Collections.Generic; using AdaptiveCards; using SBI_TeamsBot.Models; using Newtonsoft.Json.Linq; namespace SBI_TeamsBot.Services { public static class AdaptiveCardHelper { public static AdaptiveCardData CreateExampleData(AdaptiveCard adaptiveCard) { string userText = (adaptiveCard.Body[1] as AdaptiveTextBlock).Text; var choiceSet = adaptiveCard.Body[3] as AdaptiveChoiceSetInput; return new AdaptiveCardData { Question = userText, MultiSelect = choiceSet.IsMultiSelect ? "true" : "false", Option1 = choiceSet.Choices[0].Title, Option2 = choiceSet.Choices[1].Title, Option3 = choiceSet.Choices[2].Title, }; } public static AdaptiveCard CreateAdaptiveCardEditor(AdaptiveCardData exampleData) { var cardData = exampleData ?? new AdaptiveCardData(); return new AdaptiveCard("1.0") { Body = new List { new AdaptiveTextBlock("Task creation") { Weight = AdaptiveTextWeight.Bolder, }, new AdaptiveTextBlock("Which type of task would you like to create?"), new AdaptiveTextBlock() { Id = "TitleText", Text = "Title", IsVisible = false }, new AdaptiveTextInput() { Id = "Title", Placeholder = "Title", IsVisible = false, IsRequired = true}, new AdaptiveDateInput() { Id = "StartDate", IsVisible = false, IsRequired = true, Placeholder = "Start date"}, new AdaptiveDateInput() { Id = "DueDate", IsVisible = false, IsRequired = true, Placeholder = "Due date"}, new AdaptiveChoiceSetInput { Type = AdaptiveChoiceSetInput.TypeName, Id = "Choices", IsVisible = false, IsMultiSelect = bool.Parse(exampleData.MultiSelect), Value = "Choose a plan", Choices = new List { new AdaptiveChoice() { Title = exampleData.Option1, Value = exampleData.Option1Value }, new AdaptiveChoice() { Title = exampleData.Option2, Value = exampleData.Option2Value }, new AdaptiveChoice() { Title = exampleData.Option3, Value = exampleData.Option3Value }, new AdaptiveChoice() { Title = exampleData.Option4, Value = exampleData.Option4Value }, }, }, new AdaptiveActionSet() { Id = "SubmitTodoAction", IsVisible = false, Actions = new List { new AdaptiveSubmitAction { Type = AdaptiveSubmitAction.TypeName, Title = "Create ToDo task", Id = "SubmitTodo", Data = new JObject { { "Type", "todo" } }, }, }, }, new AdaptiveActionSet() { Id = "SubmitPlannerAction", IsVisible = false, Actions = new List { new AdaptiveSubmitAction { Type = AdaptiveSubmitAction.TypeName, Title = "Create Planner task", Id = "SubmitPlanner", Data = new JObject { { "Type", "planner" } }, }, }, } }, Actions = new List { new AdaptiveToggleVisibilityAction { Type = AdaptiveToggleVisibilityAction.TypeName, Title = "Todo Task", TargetElements = new List { new AdaptiveTargetElement(){ElementId = "TitleText", IsVisible = true}, new AdaptiveTargetElement(){ElementId = "Title", IsVisible = true}, new AdaptiveTargetElement(){ElementId = "StartDate", IsVisible = true}, new AdaptiveTargetElement(){ElementId = "DueDate", IsVisible = true}, new AdaptiveTargetElement(){ElementId = "SubmitTodoAction", IsVisible = true}, new AdaptiveTargetElement(){ElementId = "Choices", IsVisible = false}, new AdaptiveTargetElement(){ElementId = "SubmitPlannerAction", IsVisible = false} } }, new AdaptiveToggleVisibilityAction { Type = AdaptiveToggleVisibilityAction.TypeName, Title = "Planner Task", TargetElements = new List { new AdaptiveTargetElement(){ElementId = "TitleText", IsVisible = true}, new AdaptiveTargetElement(){ElementId = "Title", IsVisible = true}, new AdaptiveTargetElement(){ElementId = "StartDate", IsVisible = true}, new AdaptiveTargetElement(){ElementId = "DueDate", IsVisible = true}, new AdaptiveTargetElement(){ElementId = "Choices", IsVisible = true}, new AdaptiveTargetElement(){ElementId = "SubmitPlannerAction", IsVisible = true}, new AdaptiveTargetElement(){ElementId = "SubmitTodoAction", IsVisible = false}, } }, }, }; } } } ``` Those two classes will be used in the next step within our EchoBot.cs to generate and visualize the Adaptive Card in our messaging extension. ### Implement Teams Functionality The last piece of bot logic we need to add deals with the messaging extension functionality. Therefore we need to add the following lines to our EchoBot.cs to make that work: First of all we need to change the EchoBot type from an ActivityHandler to a TeamsActivityHandler and create some constants and add the constructor as follows: ```C# public class EchoBot : TeamsActivityHandler { readonly string _connectionName; private string plannerGroupId; public static string botClientID; public static string botClientSecret; private string tenantId; private string serviceUrl; public EchoBot(IConfiguration configuration) { _connectionName = configuration["ConnectionNameGraph"] ?? throw new NullReferenceException("ConnectionNameGraph"); botClientID = configuration["MicrosoftAppId"]; botClientSecret = configuration["MicrosoftAppPassword"]; tenantId = configuration["tenantId"]; serviceUrl = configuration["serviceUrl"]; } ``` Next we need to override the OnTeamsSigninVerifyStateAsync method to make sure authentication works in our messaging exentsion: ```C# protected override async Task OnTeamsSigninVerifyStateAsync(ITurnContext turnContext, CancellationToken cancellationToken) { Logger.LogInformation("Running dialog with signin/verifystate from an Invoke Activity."); // The OAuth Prompt needs to see the Invoke Activity in order to complete the login process. // Run the Dialog with the new Invoke Activity. await Dialog.RunAsync(turnContext, ConversationState.CreateProperty(nameof(DialogState)), cancellationToken); } ``` What we now need is to implement the functionality for what is called [justInTimeInstallation](https://docs.microsoft.com/en-us/microsoftteams/platform/resources/messaging-extension-v3/create-extensions?tabs=typescript#request-to-install-your-conversational-bot) to make sure we can use the messaging extension also in a team conversation where the bot is not (yet) installed. Therefore we first of all need an Adaptive Card which we will send to the user trying to open the messaging extension, requesting to install the bot in the team: ```JSON { "type": "AdaptiveCard", "body": [ { "type": "TextBlock", "text": "Looks like you haven't installed this app in this team/chat" } ], "actions": [ { "type": "Action.Submit", "title": "Continue", "data": { "msteams": { "justInTimeInstall": true } } } ], "version": "1.0", "$schema": "http://adaptivecards.io/schemas/adaptive-card.json" } ``` Next we need an override for the OnTeamsMessagingExtensionFetchTaskAsync method which will be called when the user triggers the messaging extension: ```C# // 1. Will be called when user triggers messaging extension which then calls CreateTaskModuleCommand protected override async Task OnTeamsMessagingExtensionFetchTaskAsync(ITurnContext turnContext, MessagingExtensionAction action, CancellationToken cancellationToken) { // Check if the bot has been installed in the team by getting the team rooster try { var teamsMembers = await TeamsInfo.GetMembersAsync(turnContext); } catch { // if not installed we will send out the card instructing the user to install the bot string jitCardPath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "Cards", "jitCard.json"); var jitCard = File.ReadAllText(jitCardPath, Encoding.UTF8); string jitCardJson = jitCard; var jitCardTeams = AdaptiveCard.FromJson(jitCardJson); return await Task.FromResult(new MessagingExtensionActionResponse { Task = new TaskModuleContinueResponse { Value = new TaskModuleTaskInfo { Card = new Attachment { Content = jitCardTeams.Card, ContentType = AdaptiveCard.ContentType, }, Height = 300, Width = 600, Title = "Install bot", }, }, }); } var magicCode = string.Empty; var state = (turnContext.Activity.Value as Newtonsoft.Json.Linq.JObject).Value("state"); if (!string.IsNullOrEmpty(state)) { int parsed = 0; if (int.TryParse(state, out parsed)) { magicCode = parsed.ToString(); } } var tokenResponse = await (turnContext.Adapter as IUserTokenProvider).GetUserTokenAsync(turnContext, _connectionName, magicCode, cancellationToken: cancellationToken); if (tokenResponse == null || string.IsNullOrEmpty(tokenResponse.Token)) { // There is no token, so the user has not signed in yet. // Retrieve the OAuth Sign in Link to use in the MessagingExtensionResult Suggested Actions var signInLink = await (turnContext.Adapter as IUserTokenProvider).GetOauthSignInLinkAsync(turnContext, _connectionName, cancellationToken); return new MessagingExtensionActionResponse { ComposeExtension = new MessagingExtensionResult { Type = "auth", SuggestedActions = new MessagingExtensionSuggestedAction { Actions = new List { new CardAction { Type = ActionTypes.OpenUrl, Value = signInLink, Title = "Sign in Please", }, }, }, }, }; } var accessToken = tokenResponse.Token; if (accessToken != null || !string.IsNullOrEmpty(accessToken)) { // Create Graph Client var client = new SimpleGraphClient(accessToken); // Get Group details var channel = turnContext.Activity.TeamsGetChannelId(); if (channel != null) { var members = new List(); string continuationToken = null; do { var currentPage = await TeamsInfo.GetPagedMembersAsync(turnContext, 100, continuationToken, cancellationToken); continuationToken = currentPage.ContinuationToken; members = members.Concat(currentPage.Members).ToList(); } while (continuationToken != null); TeamDetails teamDetails = await TeamsInfo.GetTeamDetailsAsync(turnContext, turnContext.Activity.TeamsGetTeamInfo().Id, cancellationToken); if (teamDetails != null) { var groupId = teamDetails.AadGroupId; plannerGroupId = groupId; //Get Plans var currentGroupPlan = await client.GetCurrentPlan(groupId); var favoritePlans = await client.GetFavoritePlans(); // Fill Adaptive Card data var exampleData = new AdaptiveCardData(); exampleData.MultiSelect = "false"; if (currentGroupPlan.CurrentPage.Count == 0) { exampleData.Option1 = favoritePlans.CurrentPage[4].Title; exampleData.Option1Value = favoritePlans.CurrentPage[4].Id; } else { exampleData.Option1 = currentGroupPlan.CurrentPage[0].Title; exampleData.Option1Value = currentGroupPlan.CurrentPage[0].Id; } exampleData.Option2 = favoritePlans.CurrentPage[0].Title; exampleData.Option3 = favoritePlans.CurrentPage[1].Title; exampleData.Option4 = favoritePlans.CurrentPage[2].Title; exampleData.Option2Value = favoritePlans.CurrentPage[0].Id; exampleData.Option3Value = favoritePlans.CurrentPage[1].Id; exampleData.Option4Value = favoritePlans.CurrentPage[2].Id; // Create and return card var adaptiveCardEditor = AdaptiveCardHelper.CreateAdaptiveCardEditor(exampleData); return await Task.FromResult(new MessagingExtensionActionResponse { Task = new TaskModuleContinueResponse { Value = new TaskModuleTaskInfo { Card = new Microsoft.Bot.Schema.Attachment { Content = adaptiveCardEditor, ContentType = AdaptiveCard.ContentType, }, Height = 600, Width = 600, Title = "Task creation", }, }, }); } } else { // Return only favorite plans without current plan as in 1:1 or group chat var favoritePlans = await client.GetFavoritePlans(); // Fill Adaptive Card data var exampleData = new AdaptiveCardData(); exampleData.MultiSelect = "false"; exampleData.Option1 = favoritePlans.CurrentPage[0].Title; exampleData.Option2 = favoritePlans.CurrentPage[1].Title; exampleData.Option3 = favoritePlans.CurrentPage[2].Title; exampleData.Option1Value = favoritePlans.CurrentPage[0].Id; exampleData.Option3Value = favoritePlans.CurrentPage[1].Id; exampleData.Option4Value = favoritePlans.CurrentPage[2].Id; // Create and return card var adaptiveCardEditor = AdaptiveCardHelper.CreateAdaptiveCardEditor(exampleData); return await Task.FromResult(new MessagingExtensionActionResponse { Task = new TaskModuleContinueResponse { Value = new TaskModuleTaskInfo { Card = new Microsoft.Bot.Schema.Attachment { Content = adaptiveCardEditor, ContentType = AdaptiveCard.ContentType, }, Height = 600, Width = 600, Title = "Task creation", }, }, }); } } return null; } ``` After the user has entered the data we need to call the CreateTaskModuleCommand which will fetch the input data and create the tasks using the SimpleGraphClient.cs: ```C# // 2. Will be called after OnTeamsMessagingExtensionFetchTaskAsync when user has entered all data in the Messaging Extension Adaptive Card private async Task CreateTaskModuleCommand(ITurnContext turnContext, MessagingExtensionAction action, CancellationToken cancellationToken) { var msg = action.MessagePayload.Body.Content; var data = action.Data; var channel = turnContext.Activity.TeamsGetChannelId(); var groupId = ""; if (channel != null) { TeamDetails teamDetails = await TeamsInfo.GetTeamDetailsAsync(turnContext, turnContext.Activity.TeamsGetTeamInfo().Id, cancellationToken); groupId = teamDetails.AadGroupId; } var subject = ((JObject)action.Data)["Title"]?.ToString(); var dueDate = ((JObject)action.Data)["DueDate"]?.ToString(); var startDate = ((JObject)action.Data)["StartDate"]?.ToString(); var type = ((JObject)action.Data)["Type"]?.ToString(); var url = turnContext.Activity.Value.ToString(); JObject jsonUrl = JObject.Parse(url); var link = jsonUrl["messagePayload"]["linkToMessage"]; var magicCode = string.Empty; var state = (turnContext.Activity.Value as Newtonsoft.Json.Linq.JObject).Value("state"); if (!string.IsNullOrEmpty(state)) { int parsed = 0; if (int.TryParse(state, out parsed)) { magicCode = parsed.ToString(); } } var tokenResponse = await (turnContext.Adapter as IUserTokenProvider).GetUserTokenAsync(turnContext, _connectionName, magicCode, cancellationToken: cancellationToken); if (tokenResponse == null || string.IsNullOrEmpty(tokenResponse.Token)) { // There is no token, so the user has not signed in yet. // Retrieve the OAuth Sign in Link to use in the MessagingExtensionResult Suggested Actions var signInLink = await (turnContext.Adapter as IUserTokenProvider).GetOauthSignInLinkAsync(turnContext, _connectionName, cancellationToken); return new MessagingExtensionActionResponse { ComposeExtension = new MessagingExtensionResult { Type = "auth", SuggestedActions = new MessagingExtensionSuggestedAction { Actions = new List { new CardAction { Type = ActionTypes.OpenUrl, Value = signInLink, Title = "Bot Service OAuth", }, }, }, }, }; } var accessToken = tokenResponse.Token; if (accessToken != null || !string.IsNullOrEmpty(accessToken)) { var client = new SimpleGraphClient(accessToken); if (type == "todo") { var body = new Beta.ItemBody { Content = msg + " - " + link }; var taskResult = await client.CreateTaskAsync(subject, dueDate, startDate, body); var todoUrl = "https://to-do.office.com/tasks/id/" + taskResult.Id + "/details"; List participants = new List(); participants.Add(new ChannelAccount(turnContext.Activity.From.Id, turnContext.Activity.From.Name)); var connectorClient = new ConnectorClient(new Uri(serviceUrl), new MicrosoftAppCredentials(botClientID, botClientSecret)); var conversationParameters = new ConversationParameters() { ChannelData = new TeamsChannelData { Tenant = new TenantInfo { Id = tenantId, } }, Members = new List() { participants[0] } }; var response = await connectorClient.Conversations.CreateConversationAsync(conversationParameters); string taskCardPath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "Cards", "todoCardTeams.json"); var r = File.ReadAllText(taskCardPath, Encoding.UTF8); string taskCardJson = r; taskCardJson = taskCardJson.Replace("replaceUrl", todoUrl ?? "", true, culture: CultureInfo.InvariantCulture); taskCardJson = taskCardJson.Replace("ReplaceTitel", taskResult.Subject.ToString() ?? "", true, culture: CultureInfo.InvariantCulture); var card = AdaptiveCard.FromJson(taskCardJson); Attachment attachment = new Attachment() { ContentType = AdaptiveCard.ContentType, Content = card.Card }; IMessageActivity cardMsg = MessageFactory.Attachment(attachment); await connectorClient.Conversations.SendToConversationAsync(response.Id, (Activity)cardMsg, cancellationToken); } if (type == "planner") { var username = turnContext.Activity.From.AadObjectId; var taskTitle = ((JObject)action.Data)["Title"]?.ToString(); var taskStartDate = ((JObject)action.Data)["StartDate"]?.ToString(); var taskDueDate = ((JObject)action.Data)["DueDate"]?.ToString(); var taskSPlanId = ((JObject)action.Data)["Choices"]?.ToString(); var planResult = await client.CreatePlannerTaskAsync(taskSPlanId, taskTitle, taskDueDate, taskStartDate, username); if (!string.IsNullOrEmpty(groupId)) { var taskUrl = "https://tasks.office.com/solviondemo.net/en-US/Home/Planner/#/plantaskboard?groupId=" + groupId + "&planId=" + planResult.PlanId + "&taskId=" + planResult.Id; List participants = new List(); participants.Add(new ChannelAccount(turnContext.Activity.From.Id, turnContext.Activity.From.Name)); var connectorClient = new ConnectorClient(new Uri(serviceUrl), new MicrosoftAppCredentials(botClientID, botClientSecret)); var conversationParameters = new ConversationParameters() { ChannelData = new TeamsChannelData { Tenant = new TenantInfo { Id = tenantId, } }, Members = new List() { participants[0] } }; var response = await connectorClient.Conversations.CreateConversationAsync(conversationParameters); string taskCardPath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "Cards", "plannerCardTeams.json"); var r = File.ReadAllText(taskCardPath, Encoding.UTF8); string taskCardJson = r; taskCardJson = taskCardJson.Replace("replaceUrl", taskUrl ?? "", true, culture: CultureInfo.InvariantCulture); taskCardJson = taskCardJson.Replace("ReplaceTitel", planResult.Title.ToString() ?? "", true, culture: CultureInfo.InvariantCulture); var card = AdaptiveCard.FromJson(taskCardJson); Attachment attachment = new Attachment() { ContentType = AdaptiveCard.ContentType, Content = card.Card }; IMessageActivity cardMsg = MessageFactory.Attachment(attachment); await connectorClient.Conversations.SendToConversationAsync(response.Id, (Activity)cardMsg, cancellationToken); } else { List participants = new List(); participants.Add(new ChannelAccount(turnContext.Activity.From.Id, turnContext.Activity.From.Name)); var connectorClient = new ConnectorClient(new Uri(serviceUrl), new MicrosoftAppCredentials(botClientID, botClientSecret)); var conversationParameters = new ConversationParameters() { ChannelData = new TeamsChannelData { Tenant = new TenantInfo { Id = tenantId, } }, Members = new List() { participants[0] } }; var response = await connectorClient.Conversations.CreateConversationAsync(conversationParameters); var personalMessageActivity = MessageFactory.Text($"I've created a new Planner task with the title **" + planResult.Title.ToString() + "** in the Plan you have chosen"); await connectorClient.Conversations.SendToConversationAsync(response.Id, personalMessageActivity); } } } return null; } ``` The last piece here is an override for the OnTeamsMessagingExtensionSubmitActionAsync method which we will need to call our CreateTaskModuleCommand: ```C# protected override async Task OnTeamsMessagingExtensionSubmitActionAsync(ITurnContext turnContext, MessagingExtensionAction action, CancellationToken cancellationToken) { // check if we previousy requested to install the bot - if true we will present the messaging extension if (action.Data.ToString().Contains("justInTimeInstall")) { return await OnTeamsMessagingExtensionFetchTaskAsync(turnContext, action, cancellationToken); } else { switch (action.CommandId) { case "createTaskModule": // This command will first call OnTeamsMessagingExtensionFetchTaskAsync and then CreateTaskModuleCommand return await CreateTaskModuleCommand(turnContext, action, cancellationToken); default: throw new NotImplementedException($"Invalid CommandId: {action.CommandId}"); } } } ``` ## Create Teams App Now that the bot is ready, we can create our new Microsoft Teams application using the Teams App Studio. Within there we well connect our Bot Channels Registration service we created earlier and also add a messaging extension command with the name **createTaskModule** exactly as we have used in our EchoBot.cs: {{< figure src="/images/061-09.png" >}} From the Bots section, make sure to add personal, team and group as a scope so users can use your messaging extension in 1:1 & group chats and teams conversations. In order to create the messaging extension correctly, you'll need to select the same bot as before and then add a new command like this: {{< figure src="/images/061-10.png" >}} {{< figure src="/images/061-11.png" >}} {{< figure src="/images/061-12.png" >}} From the Domains and permissions section also add token.botframework.com to the list of valid domains to make authentication work: {{< figure src="/images/061-13.png" >}} After installing the bot for your users you can test it (make sure you added the Teams channel to your Bot Channels registration in the Azure portal) and you should be greeted immediately by the bot: {{< figure src="/images/061-14.png" >}} Now if you click the ellipsis (...) on a message and select more actions you should see your "Creat Task" command there: {{< figure src="/images/061-15.png" >}} If you click that the first time you will be prompted to authenticate: {{< figure src="/images/061-16.png" >}} After you have signed, in the bot will show the Adaptive Card where you can select to create either a To Do task or a Planner task: {{< figure src="/images/061-17.png" >}} If you click on **Todo Task** you can insert the title, start and due date of your to be created task: {{< figure src="/images/061-18.png" >}} After hitting **Create ToDo Task** the bot will create the task in your ToDo task list and send you back a message that the task creation has been done successfully: {{< figure src="/images/061-19.png" >}} Now if you click on **Open Task** you will be directly getting to that newly created task in To Do: {{< figure src="/images/061-20.png" >}} Now even cooler is the functionality in a team conversation, where you have a Planner Plan attached. If I select the **Create Task** action again on a conversation in a team and then create a new Planner task, it will let me select my favorite Planner plans as well as the Planner plan for this particular team and the bot will send you a 1:1 direct message when the creation has been finished: {{< figure src="/images/061-21.png" >}} > The code is published in the Bot Builder Community GitHub repo and can be found [here](https://github.com/BotBuilderCommunity/botbuilder-community-dotnet/tree/develop/samples/Teams%20Graph%20Messaging%20Extensions) (feel free to use or contribute) ## Conclusion NOTE: This code is definetly NOT production ready but you can of course use it and modify it to use it for your use case (but please don't blame me if there are bugs 😁). The things I learned while developing this bot are: - Authentication is still a bit tricky with so many components playing together (Azure AD, Graph, Teams, Bot Framework, ...) - Adaptive Cards come in handy when you want to build your own UI for your messaging extensions - Notifying users using a 1:1 chat message from a team conversation is a bit hacky as of now - The Microsoft Bot Framework is awesome 👍 --- title: "Bot Framework Composer Series - 1 - Use Adaptive Cards" url: https://bisser.io/bot-framework-composer-series-1-use-adaptive-cards/ markdown: https://bisser.io/bot-framework-composer-series-1-use-adaptive-cards/index.md date: 2020-04-24 lastmod: 2020-04-24 tags: AI, Bot Framework categories: AI, Bot Framework description: "First part of a blo series dealing with Microsoft Bot Framework Composer. This post is all about how to handle Adaptive Cards right" status: legacy reading_time_min: 5 --- ## The Problem As I was using [Microsoft Bot Framework Composer](https://github.com/microsoft/BotFramework-Composer) quite heavily in the past, I built a lot of bots and dialogs with it. Therefore, I want to take the opportunity to share all my learnings I had with this awesome tool with everyone in a blog series. In this part I want to cover something, which I often found difficult to handle in my bot: rich attachments like Adaptive Cards, as I was struggling a bit on how to implement them correct. As Composer is still in preview by the time of writing, it is totally understandable that some things are not yet shipped or documented. Therefore, this post should outline on how to use Adaptive Cards within Composer for your bots. ## Implementing Adaptive Cards in BF Composer First of all we will create a brand new bot using the Echo Bot template: {{< figure src="/images/060-bot-creation.png" >}} Now that our new bot has been created, we can take a look at the **Main** dialog, which essentially has 2 triggers: * Unknown intent * Action to perform when user input is unrecognized and if none of the 'on intent recognition' triggers match recognized intent. * Greeting * Handle the events fired when a user begins a new conversation with the bot. Now when I built a bot, the first thing I do is that I want my bot to greet the user upon starting the conversation. But I don't want to do that using text only, but I want to create a wow-effect using an Adaptive Card. This way, you can share so much more details and information with your users than just sending some greeting text only. So in order to achieve this, we need to add an Adaptive Card. To create an Adaptive Card it's probably the easiest way to use the [Adaptive Cards Designer](https://adaptivecards.io/designer/) as you can design your card using a visual interface rather than writing JSON: {{< figure src="/images/060-card-designer.png" >}} Now that we have our card, we need to switch back to Composer and edit our Bot responses, which will look like this in the beginning: {{< figure src="/images/060-bot-responses-before.png" >}} The Bot Responses let's us actually manipulate the .lg file used for the bot. If you want to dig deeper into what's possible with that just visit the [Composer docs](https://github.com/microsoft/BotBuilder-Samples/blob/master/experimental/language-generation/docs/lg-file-format.md). In the edit mode of our Bot responses, we then need to paste in our Adaptive Card JSON like the following: ```json # adaptivecardjson - ``` { "type": "AdaptiveCard", "id": "NewUserGreeting", "backgroundImage": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAKwAAACeCAYAAACvg+F+AAAABGdBTUEAALGPC/xhBQAAAAFzUkdCAK7OHOkAAAAJcEhZcwAAFiUAABYlAUlSJPAAAAAhdEVYdENyZWF0aW9uIFRpbWUAMjAxOTowMzoxMyAxOTo0Mjo0OBCBEeIAAAG8SURBVHhe7dJBDQAgEMCwA/+egQcmlrSfGdg6z0DE/oUEw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phSTEsKYYlxbCkGJYUw5JiWFIMS4phCZm52U4FOCAVGHQAAAAASUVORK5CYII=", "body": [ { "type": "Container", "items": [ { "type": "Image", "url": "https://bisserio.blob.core.windows.net/img/bot_bisserio_card.png", "size": "Stretch" } ] }, { "type": "Container", "spacing": "None", "backgroundImage": "https://bisserio.blob.core.windows.net/img/sidebar_bg_blog_bisser_io.png", "items": [ { "type": "TextBlock", "id": "title", "spacing": "Medium", "size": "Large", "weight": "Bolder", "color": "Light", "text": "Hey, I'm Steve - your digital assistant", "wrap": true }, { "type": "TextBlock", "id": "body", "size": "Medium", "color": "Light", "text": "You can ask me question about everything Conversational AI or about Stephan 😉", "wrap": true } ] } ], "actions": [ { "type": "Action.Submit", "title": "Help", "data": "Help", "iconUrl": "https://img.icons8.com/ios/50/000000/help.png", "id": "helpBtn" } ], "$schema": "http://adaptivecards.io/schemas/adaptive-card.json", "version": "1.0", "speak": "Hey, I'm Alfred - Stephan Bisser's digital assistant. You can ask me question about everything Conversational AI or even about Stephan himself", "fallbackText": "This is a default fallback in case I cannot render your card" } > Don't forget the backticks at the end (you can remove the comment though) ``` ``` > Please just remove the string _"> Don't forget the backticks at the end (you can remove the comment though)"_ from the response mentioned above to avoid any errors when testing and make sure that only the 3 backticks are there at the end... Now your bot responses should look like this: {{< figure src="/images/060-bot-responses-card.png" >}} Next we need to a function to our Bot Responses which actually reads the json we pasted in and creates an attachment from that: ```md ## AdaptiveCard [Activity Attachments = ${json(adaptivecardjson())} ] ``` Now the Bot responses should look like this: {{< figure src="/images/060-bot-responses-final.png" >}} Now that everything is in place, we can go back to our Main dialog and edit the Greeting part. In there we want to send a response to each new user. The response we want to send is the Adaptive Card we added to our Bot responses, therefore we only need to call the function we created in the previous step which generates the attachment for us by inserting the following as a response: ```md -${AdaptiveCard()} ``` The dialog should by now look like this: {{< figure src="/images/060-greeting-dialog.png" >}} That's mainly it and we can go ahead and test our bot to see if the Adaptive Card will be sent upon starting the conversation. Simply click **Start Bot** from the top right corner and when the bot has been started click **Test in Emulator** to open up your Bot Framework Emulator: {{< figure src="/images/060-bot-test.png" >}} ## The Conclusion Now with this post, we learned how to insert and use Adaptive Cards in Bot Framework Composer and it's language generation files correct. If you want to know more on how to send other types of cards I would advise to take a look at the [docs](https://github.com/microsoft/BotFramework-Composer/blob/stable/docs/how-to-send-cards.md) as they cover all different card types which are supported. Here are the other parts of my Bot Framework Composer series: - [Bot Framework Composer Series - 2 - Property management](https://bisser.io/bot-framework-composer-series-2-property-management/) As this was the first of a many to come blog post in a series around Bot Framework Composer, I hope you'll find that useful! If you have any topics you want me to cover in a future post, just drop me a message... Happy composing 😀 --- title: "Building Bot Framework Analytics Dashboards using Application Insights" url: https://bisser.io/building-bot-framework-analytics-dashboards-using-application-insights/ markdown: https://bisser.io/building-bot-framework-analytics-dashboards-using-application-insights/index.md date: 2020-04-20 lastmod: 2020-04-20 tags: AI, Bot Framework categories: AI, Bot Framework description: "If you are building bots which are not used - you are not building bots! So analyze your bot's performance with these little tricks" status: legacy reading_time_min: 5 --- ## Objectives If you are building chatbots, the one thing you always need to keep in mind is the build useful bots. In most cases you will start by implementing a more or less simple QnA bot utilizing [QnA Maker](https://qnamaker.ai) in your bot to answer commonly asked questions. And even with such a simple bot, which is "only" answering questions stored in a knowledge base, you would likely need to have some sort of analytics service, which indicates if your bot is performing well or not. > If you don't build bots that are actually used, you are not building bots Therefore, this blog post will walk you through the steps needed in order to add analytics via [Azure Application Insights](https://aka.ms/appinsights-overview) to your bot to track the bot's performance. ## Prepare the Bot First of all, we to provision a new Application Insights instance in our Azure environment. This can be simply done by searching for "Azure Application Insights" in the Azure portal and create a new service instance. Atfer this has been provisioned, we need to grab the "Instrumentation Key" from the App Insights pane which we'll later on use within the bot to establish a communication between this service and our bot: {{< figure src="/images/059_appInsightsKey.png" >}} Now after this you would need to follow [this link](https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-telemetry-qnamaker?view=azure-bot-service-4.0) to add all neceassary components to your bot to enable the logging into App Insights. After doing so you can go to your App Insights service in the Azure portal and execute this query from the "Logs" section: ```SQL customEvents | where name == 'QnaMessage' | extend answer = tostring(customDimensions.answer) | summarize count() by answer ``` After executing this query, you should see the following result (with your answers): {{< figure src="/images/059_firstQuery.png" >}} But as you might guess, there is something missing from the above results. It would be far better if we would not only get the answer in there, but also the question, so we can build a list which holds each question and the corresponding answer. To achieve that, we need to adapt our bot a little as [this link](https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-telemetry-qnamaker?view=azure-bot-service-4.0#modifying-or-extending-the-default-event-properties) indicates. The first thing we need to add a new class *DemoQnAMaker.cs* in our project, which should look like this: ```C# public class DemoQnAMaker : QnAMaker { public DemoQnAMaker( QnAMakerEndpoint endpoint, QnAMakerOptions options = null, HttpClient httpClient = null, IBotTelemetryClient telemetryClient = null, bool logPersonalInformation = false) : base(endpoint, options, httpClient, telemetryClient, logPersonalInformation) { } protected override async Task OnQnaResultsAsync(QueryResult[] queryResults, Microsoft.Bot.Builder.ITurnContext turnContext, Dictionary telemetryProperties = null, Dictionary telemetryMetrics = null, CancellationToken cancellationToken = default(CancellationToken)) { var eventData = await FillQnAEventAsync(queryResults, turnContext, telemetryProperties, telemetryMetrics, cancellationToken).ConfigureAwait(false); // Get question var userquestion = turnContext.Activity.Text.ToString(); // Add new property eventData.Properties.Add("UserQuestion", userquestion); // Log QnAMessage event TelemetryClient.TrackEvent(QnATelemetryConstants.QnaMsgEvent, eventData.Properties, eventData.Metrics); } } ``` After that, we need to change the code in the *DemoBot.cs* (or however your bot logic file is called) to use this new class instead of the default QnAMaker class: ```C# var qnaMaker = new DemoQnAMaker(new QnAMakerEndpoint { KnowledgeBaseId = _configuration["QnAKnowledgebaseId"], EndpointKey = _configuration["QnAEndpointKey"], Host = _configuration["QnAEndpointHostName"] }, null, httpClient, _telemetryClient); ``` This implementation now makes sure, that all activities in which QnA Maker is involved, that all questions from the users will also be sent to App Insights for further analytics. Now after your bot is published to Azure, you can go ahead and test the bot in whatever channel you like, which will trigger App Insights to collect the telemetry data. ## Create Queries and a Dashboard Now the bot is ready and sends telemetry data to our App Insights service, we can create and execute queries in the Azure portal. To give you an idea which queries you can run, the following queries can be used to visualize different information: ### Users in last 14 days This query basically visualizes the number of active users in the last 14 days in a linechart: ```SQL Number of users: let queryStartDate = ago(14d); let queryEndDate = now(); let groupByInterval = 1d; customEvents | where timestamp > queryStartDate | where timestamp < queryEndDate | summarize uc=dcount(user_Id) by bin(timestamp, groupByInterval) | render timechart ``` ### QnA Results for all users If you want to see all questions and answers along with the count of occurences, you can use the following query: ```SQL customEvents | where name == 'QnaMessage' | extend answer = tostring(customDimensions.answer) | extend UserQuestion = tostring(customDimensions.UserQuestion) | where UserQuestion != "" | summarize count() by answer, UserQuestion ``` ### QnA Results by user The following query enhances the previous query by also adding the userId so you can filter questions and answers by the userId. ```SQL customEvents | where name == 'QnaMessage' | extend UserQuestion = tostring(customDimensions.UserQuestion) | extend answer = tostring(customDimensions.answer) | extend user = tostring(user_Id) | where UserQuestion != "" | summarize by UserQuestion, answer, user ``` **NOTE:** This query requires you to add the following line to your DemoQnAMaker.cs to also send the userId to App Insights: ```C# eventData.Properties.Add("UserName", username); ``` These queries can now be added to a dashboard just from the query explorer: {{< figure src="/images/059_addQuerytoDashboard.png" >}} After you have added the query results to your dashboard, you can customize your dashboard to make it look more appealing, like the following for example: {{< figure src="/images/059_customDashboard.png" >}} ## Conclusion Now that you have enabled your bot to log information into Application Insights, you can go ahead and build beautiful dashboards, which could be shared with users, like the bot admins or the content creators, so they know how the bot is performing and where to tune something to make sure that the end users are happy and adopt your bot (as this is the most important thing at the end of the day). Also please make sure to stay GDPR compliant and do not store any user data against the user's will. Always tell the users that you are storing conversation and user data for analytics purposes. Happy analyzing 🐱‍🏍 --- title: "Bot Framework CLI for LUIS Management" url: https://bisser.io/bot-framework-cli-for-luis-management/ markdown: https://bisser.io/bot-framework-cli-for-luis-management/index.md date: 2020-03-06 lastmod: 2020-03-06 tags: AI, Bot Framework description: "I wanted to automate the creation of my bots and the necessary services a bit. Therefore, I first took a look at what can be done in order to save time when working with LUIS. This post may be turning out to be the first of a many to come series about automating Bot Framework development and depl..." status: legacy reading_time_min: 4 --- ## Problem Statement I wanted to automate the creation of my bots and the necessary services a bit. Therefore, I first took a look at what can be done in order to save time when working with LUIS. This post may be turning out to be the first of a many to come series about automating Bot Framework development and deployment. But first let's take a look at how one can leverage the [BotFramework CLI](https://github.com/microsoft/botframework-cli) in order to create, train and publish LUIS apps using only the CLI and not the portal. ## Create new LUIS app and import .lu file using BF CLI This section will outline how to create a new LUIS app from a .lu file as well as training it and publishing it all from the [BotFramework CLI](https://github.com/microsoft/botframework-cli). ### Convert .lu to .json file At first we need to create a new .lu file or reuse one which has already been created. For sample .lu files simply check out [this link](https://github.com/microsoft/botbuilder-tools/blob/master/packages/Ludown/examples). Typically a .lu file looks as follows: ``` > LUIS application information > !# @app.name = bisserioLUISapp > !# @app.desc = LUIS app for bisserio > !# @app.versionId = 0.1 > !# @app.culture = en-us > !# @app.luis_schema_version = 6.0.0 > Help intent and its utterances # Help - help - i need help - please help - can you please help - what do you do - what is this bot for > Greeting intent and its utterances # Greeting - hi - hey - hello - good afternoon - good evening - good morning > BuySurface intent and its utterances # BuySurface - How can I buy {ProductType=Surface PRO} - I want to buy {ProductType=Surface PRO} - I want to buy {ProductType=Surface laptop} - Can I buy {ProductType=Surface PRO} online ``` After all intents and utterances have been setup correctly int that file we need to convert this .lu file to a .json file to be able to import it later on by using the following command: ``` bash bf luis:convert --in "pathToLuFile" --culture "cultureCodeOfLuisApp" --out "pathToJsonFile" ``` > NOTE: The culture code needs to be in the format like "en-us" or "de-de" to be correct. [See here](https://botbuilder.myget.org/feed/botframework-cli/package/npm/@microsoft/botframework-cli#bf-luisconvert) for more details on this command. ### Import new LUIS application Next we can import the previously created json file and create a new LUIS app from that json file using the following command: ``` bash bf luis:application:import --endpoint "https://.api.cognitive.microsoft.com" --subscriptionKey "LuisSubscriptionKey" --name "LuisAppName" --in "pathToJsonFile" ``` > NOTE: The subscriptionKey needs to be the authoringKey which you can get either from the [Azure Portal](https://portal.azure.com) or from the [LUIS Portal](https://luis.ai). The region needs to be the Azure region you deployed your LUIS authoring service into. [See here](https://botbuilder.myget.org/feed/botframework-cli/package/npm/@microsoft/botframework-cli#bf-luisapplicationimport) for more details on this command. This will create a new LUIS app and you should get the according App ID as a console output like this: **App successfully imported with id 2d79d034-8d67-46e7-a20e-ca087640d872.** Now copy this id as you will need that later on. ### Show LUIS app To verify if the import has been correct we can use the following command to show the LUIS app: ``` bash bf luis:application:show --appId "appIdFromPreviousCommand" --endpoint "https://.api.cognitive.microsoft.com" --subscriptionKey "LuisSubscriptionKey" ``` [See here](https://botbuilder.myget.org/feed/botframework-cli/package/npm/@microsoft/botframework-cli#bf-luisapplicationshow) for more details on this command. ### Train LUIS app After the application has been imported correctly we can train it before we publish it by executing this command: ``` bash bf luis:train:run --appId "appIdFromPreviousCommand" --versionId "0.1" --endpoint "https://.api.cognitive.microsoft.com" --subscriptionKey "LuisSubscriptionKey" ``` [See here](https://botbuilder.myget.org/feed/botframework-cli/package/npm/@microsoft/bf-luis-cli#bf-luistrainrun) for more details on this command. After that we can verify if the train command executed successfully by executing this command: ``` bash bf luis:train:show --appId "appIdFromPreviousCommand" --versionId "0.1" --endpoint "https://.api.cognitive.microsoft.com" --subscriptionKey "LuisSubscriptionKey" ``` [See here](https://botbuilder.myget.org/feed/botframework-cli/package/npm/@microsoft/botframework-cli#bf-luistrainshow) for more details on this command. This should output something like the following: ``` json [ { "modelId": "2881763e-7d05-4e01-9161-64efc22eaeaa", "details": { "statusId": 0, "status": "Success", "exampleCount": 22, "trainingDateTime": "2020-03-06T12:01:27.000Z" } }, { "modelId": "c2f5279a-4fb4-45bd-82ac-fc7ea5c64d5b", "details": { "statusId": 0, "status": "Success", "exampleCount": 22, "trainingDateTime": "2020-03-06T12:01:27.000Z" } }, { "modelId": "b867a8dd-2a0c-46f6-8a4c-730cce87920a", "details": { "statusId": 0, "status": "Success", "exampleCount": 22, "trainingDateTime": "2020-03-06T12:01:27.000Z" } }, { "modelId": "ddc2cfe2-a545-42c7-9093-679aeff3cc7a", "details": { "statusId": 0, "status": "Success", "exampleCount": 22, "trainingDateTime": "2020-03-06T12:01:27.000Z" } } ] ``` ### Publish LUIS app If the train operation has been successful, we can publish the LUIS app by executing this command: ``` bash bf luis:application:publish --endpoint "https://.api.cognitive.microsoft.com" --subscriptionKey "LuisSubscriptionKey" --versionId "0.1" --appId "appIdFromPreviousCommand" ``` [See here](https://github.com/microsoft/botframework-cli/tree/master/packages/luis#bf-luisapplicationpublish) for more details on this command. The output of this will look as follows: ``` json { "versionId": "0.1", "isStaging": false, "endpointUrl": "https://westeurope.api.cognitive.microsoft.com/luis/v2.0/apps/2d79d034-8d67-46e7-a20e-ca087640d872", "region": "westeurope", "assignedEndpointKey": null, "endpointRegion": "westeurope", "failedRegions": "", "publishedDateTime": "2020-03-06T12:02:09Z", "directVersionPublish": false } ``` Now your LUIS app should be ready to use for your bot. --- title: "Speaking at SharePoint Conference 2020 in Las Vegas" url: https://bisser.io/speaking-at-sharepoint-conference-2020-in-las-vegas/ markdown: https://bisser.io/speaking-at-sharepoint-conference-2020-in-las-vegas/index.md date: 2020-01-31 lastmod: 2020-01-31 tags: AI, Bot Framework description: "Speaking at SharePoint Conference 2020 in Las Vegas" status: legacy reading_time_min: 4 --- I am delighted so say, that I got accepted for this year’s SharePoint conference 2020 in Las Vegas. The event will take place from 17th to 22nd of May in total. Standard sessions will be presented May 19-21 and full day workshops will take place May 17, 18 and 22. This conference is a must-see if you want to learn anything about Office 365, SharePoint and OneDrive as there will be more than 150 speakers (Microsoft employees as well as industry-leaders and MVPs), more than 200 sessions (ranging from development focus to targeting IT pros and business users) and more than 20 full-day workshops, which allow you to get a deep dive into one specific topic. It’s a real honor for me to be part of this great conference, along with many leaders like Jeff Teper, Dan Holme or Vesa Juvonen, who will share their knowledge and their stories and vision about the products they are responsible for. If you want to hear the latest trends and information on the newest products & features, make sure to book your ticket now. My session with the title “Building intelligent bots for your SharePoint Intranet: Best Practices from the trenches” will be a 60-minute session focusing on the topic developing solutions with the SharePoint Framework, bots and more primarily. In more detail, the session description is: _Microsoft offers multiple services to build a chatbot inside your modern workplace, however, if you start building bots without knowing the ins and outs of each platform, you might hit a wall fast. Join this session to learn what works, and what doesn’t when building a chatbot for Microsoft 365, especially for SharePoint (Online) and Teams powered by the Microsoft Graph. This session will cover everything from the Microsoft Bot Framework, Cognitive Services and the Conversational AI Platform to add value to your Microsoft 365 environment._ {{< figure src="/images/057_cover.webp" >}} Everyone attending this session will learn something about the real-world use cases and best practices for designing and implementing chatbot projects from the field. As an AI MVP specializing in bots, this session primarily focuses on the Microsoft Conversational AI platform, specially the Microsoft Bot Framework and Microsoft Azure Cognitive Services, to build sophisticated chatbots, which can then be integrated into the SharePoint (Online) based intranet. If you want to be part of this year’s event you can simply register using [this link](https://sharepointna.com/#!/register?utm_term=BISSER) and make sure to use the discount code **BISSER** to get a 50$ discount on your ticket. When registering you can choose between a lot of cool offers, where you can either get gift cards or surface devices and a XBOX ONE as a gift, so be sure to check out those offers (Seats are limited): Show and Package Options | Early Early Bird | Early Bird | After Feb. 24th, 2020 -------------------------|------------------|------------|---------------------- | Jan. 21st - Feb. 3rd, 2020 | Feb. 4th - Feb. 24th, 2020 | Full Conference Only | $1,799 | $1,849 | $1,899 SHOW PACKAGE - 1 WORKSHOP  | $2,198 | $2,248 | 2299 Includes full conference registration,  | Receive a choice of Cortana Enabled Amazon Echo or a $100 Hotel Gift Card | Receive a $100 Hotel Gift Card | ONE optional full-day workshop | | | COMPLETE PACKAGE - 2 WORKSHOPS  | $2,647 | $2,697 | $2,747 Includes full conference registration,  | Receive a choice of Cortana Enabled Amazon Echo, a $150 Hotel Gift Card, Surface Headphones or XBOX ONE S | Receive a $150 Hotel Gift Card | TWO optional full-day workshops | | | COMPLETE PACKAGE - 3 WORKSHOPS  | $3,096 | $3,146 | $3,199 Includes full conference registration,  | Receive a choice of Enabled Amazon Echo, a $200 Hotel Gift Card, XBOX ONE S, XBOX ONE X, Surface Headphones or Surface Go | Receive a $200 Hotel Gift Card | THREE optional full-day workshops | | | Pre-conference OR Post-conference | $749 | $799 | $799 Optional full-day Workshop | | | In case you want to get the latest news and trends already before the conference, tune in to SharePoint TV. SharePointTV will be broadcast over Twitch Streaming on most Wednesdays from 10-11am. Twitch Streaming is the latest video presentation platform taking the IT world by storm. Twitch was initially built as a platform for gamers to share and interact with their community. It didn’t take long for this easy access platform to grow beyond the gamer community. It is now a very popular platform for developers and the IT communities. After successfully introducing Virtual DEVintersection we launched SharePointTV at SPC19. In just 3 days we signed on 300+ followers and received 14K plus unique views. Programing is streamed live and then posted to the SPC YouTube Channel for on demand viewing. SharePointTV will be a one stop shop for training and discussions about all things SharePoint, Teams and Microsoft 365. Hosted by Vlad Catrinescu each week he will invite exciting guests from Microsoft or the community to discuss that’s new and what’s coming in the world SharePoint and related technologies. The Twitch platform allows attendees to be a part of the conversation with live chat interaction. --- title: "Local Typescript Bot Auto-Reloading using Nodemon" url: https://bisser.io/local-typescript-bot-auto-reloading-using-nodemon/ markdown: https://bisser.io/local-typescript-bot-auto-reloading-using-nodemon/index.md date: 2019-11-20 lastmod: 2019-11-20 tags: AI, Bot Framework description: "Local Typescript Bot Auto-Reloading using Nodemon" status: legacy reading_time_min: 2 --- ## Problem Statement When developing bots using the [Bot Framework SDK v4 for JavaScript](https://github.com/Microsoft/botbuilder-js) you certainly want to develop your bots locally using TypeScript, instead of doing the whole development within the Azure poral using the online code editor. But what might be annoying when doing that is that by default, you would need to run `npm start` for starting the bot. The only problem with that is, that you would need to restart the command again if you changed something in your code to be able that you can test the bot with the latest changes from the Bot Framework emulator. ## Solution To avoid this, I was seeking for a way of auto-restarting the bot in the background upon every change/save I do from my code base. And the most easiest way of doing that, is to use a tool called [Nodemon](https://nodemon.io/). So here are the steps you need to do for your bot project to be auto-reload enabled: ### Install Nodemon & ts-node Assuming you have your bot project already setup locally, we need to install two dependencies: ```shellnpm install -g nodemon npm i install -g ts-node ``` ### Create nodemon.json Within your project's root, you now need to create a new file called "nodemon.json" with the following content (make sure that the path to your index.ts file is correct): ```JSON{ "watch": ["src"], "ext": "ts", "exec": "ts-node ./src/index.ts" } ``` ### Run nodemon Now you should be able to run the `nodemon` command within your favorite shell from your project's root and should see this screen: {{< figure src="/images/056_nodemon_run.png" >}} If you have setup everything correctly, try to go ahead and change something in your code and just test the bot with the emulator, without restarting node.js. You should immediately see your changes in your bot without the need of reloading everything, which saves a lot of time and effort! Hope this makes your development cycle a bit easier as you don't need to restart everything after changing just a few lines of code... Happy coding! --- title: "Integrate QnA Maker Bots - Part 1 - Public Website" url: https://bisser.io/integrate-qna-maker-bots-part-1-public-website/ markdown: https://bisser.io/integrate-qna-maker-bots-part-1-public-website/index.md date: 2019-11-05 lastmod: 2019-11-05 tags: AI, Cognitive Services description: "Integrate QnA Maker Bots - Part 1 - Public Website" status: legacy reading_time_min: 6 --- ## Problem Statement Nowadays, it's quite common to add a "virtual assistant" (a.k.a chatbot) to your public facing website. Almost every organization, especially the ones dealing with endusers (B2C market-oriented organizations) try to add value to their websites in form of a bot. And Microsoft has done a pretty good job, making it easy to build a bot, with little effort and almost no coding. But unfortunately, there is not a complete guide provided, on how to create the bot from the very first click until it can be deployed on to a public website. So this post, shall be one of a few in a blog post series, where I will tackle, the various deployment/integration options you have for your bot (like a webiste, like a SharePoint Online page, Microsoft Teams, ...). ## QnA Maker and Azure Bot Creation So the first thing we need is our bot, of course. So in order to create a bot, you need an Azure Subscription, where you have the permissions to create new resources in (but I won't go through that step, as this is very well documented in the [docs](https://docs.microsoft.com/en-us/azure/cognitive-services/qnamaker/quickstarts/create-publish-knowledge-base#prerequisite?WT.mc_id=AI-MVP-5003073). ### QnA Maker Knowledge Base In order to create a knowledge base, we need to head over to the Azure Portal to create a new QnA Maker resource and fill out the details to fit your needs as shown here: {{< figure src="/images/055_qnamaker_creation.png" >}} Next up, we need to go to the [QnA Maker portal](https://qnamaker.ai), sign in with the account you used for creating the Azure resource and go to the "Create a knowledge base" page to create a new knowledge base along with the necessary details: {{< figure src="/images/055_KB_creation.png" >}} So in my case, I enabled the [multi-turn extraction feature](https://docs.microsoft.com/en-us/azure/cognitive-services/qnamaker/how-to/multiturn-conversation#create-a-multi-turn-conversation-from-documents-structure?WT.mc_id=AI-MVP-5003073) for my knowledge base and added a URL to a PDF to extract the questions and answers from. Another quite new feature is that you now can choose the actual language of the KB, which affects the chit-chat and the ranking algorithms of the KB which you create. Now upon entering all the details, you can go ahead and create your knowledge base. After the KB has been created, you can either alter the QnA pairs in there or if you are happy with the KB contents, you can go to the Publish page and publish your QnA Maker knowledgebase: {{< figure src="/images/055_KB_publishing01.png" >}} And so after it has been published, the API endpoint gets available and is now ready to be consumed by your application/bot: {{< figure src="/images/055_KB_publishing02.png" >}} ### Azure Bot Creation So the next obvious step would be to create our bot. In the past we would need to create a bot from the Azure portal first and then add some lines of code to call QnA Maker to get the right answers for the users' inputs, but this, fortunately, is a thing of the past. The simplest way of creating a QnA Maker integrated bot is to hit the "Create Bot" button right from the Publish page in the QnA Maker portal. So when you click that button, you will be redirected to the Azure portal and the creation pane of a new web app bot is shown. Now this creation pane is somewhat different compared to the one when you create a new bot from the Azure portal, as it has some attributes already pre-filled, like your QnA Auth Key and the regions and so on, so you basically just need to hit Create: {{< figure src="/images/055_qna_bot_creation.png" >}} After a couple of minutes, your new Azure Web App Bot should be created and you should be able to test it in the webchat right from the Azure Portal. So now the bot is ready, we can go to the next step and add it to our website. ## Bot Integration Into Website If you have a public facing website, it's quite easy to integrate a Azure bot into your website. First thing we need is to go to the Channels section in out bot's menu pane within the Azure portal and click on the "Direct Line" channel icon to enable it: {{< figure src="/images/055_bot_webchat_integration01.png" >}} Upon the "Configure Direct Line" page opens, copy one of the two secret keys and click done. Now you will be redirected back to the channels page, where you need to click "Get bot embed codes" in order to get the most easiest way of integrating a bot into a website - an IFrame embed code, which may look as follows: ```html ``` So what you actually would need to do now is to replace the two placeholders in there with your bot's name and your Direct Line secret key you copied earlier and then you could integrate that line of html code into your website to enabled the bot webchat in your site. ### Webchat Styling But in many cases the default styling of the webchat you get with that embed code isn't what you might be looking for, so we need to go another path of integrating the webchat component in your site. In one of my [previous posts](https://bisser.io/bot-framework-v4-webchat-styling-options/) I went through the options you have in terms of styling the Bot Framework webchat component, so we'll follow along and integrate the bot into our website with slightly more lines of code, but have the advantage of adding custom styling (colors, avatars, fonts, ...) to our webchat component on our site. So a good example would be to add the following lines of code to your HTML code of your website to integrate the bot along with custom styling options for the bot's chat window and add your own Direct Line secret in there as well as change, the pictures used in the webchat: ```html
``` So after you have added these lines of code to your website, the webchat component should look similar to this one: {{< figure src="/images/055_bot_webchat_integration.png" >}} So now you can play around with the styling options mentioned [here](https://bisser.io/bot-framework-v4-webchat-styling-options/) to add your favorite colors and other visuals to the webchat component on your public website to render the bot's webchat accordingly and make it fit to your website's design guideline. ## Summary/Outlook If you walk through the steps mentioned above, you will have a working bot deployed on your website within a couple of moments, so this proves, that the actual creation of a simple QnA bot as well as the webchat integration of that bot in a website via HTML and JavaScript is not a hard task. In the next part of this blog post series, we'll take a look at another channel for integrating the bot into and do some sort of styling in there, so stay tuned... --- title: "Ignite 2019 News on the Conversational AI Platform" url: https://bisser.io/ignite-2019-news-on-the-conversational-ai-platform/ markdown: https://bisser.io/ignite-2019-news-on-the-conversational-ai-platform/index.md date: 2019-11-04 lastmod: 2019-11-04 tags: AI, Bot Framework description: "Ignite 2019 News on the Conversational AI Platform" status: legacy reading_time_min: 4 --- Each year Microsoft is inviting intersted people from all over the globe to a big show, called [Microsoft Ignite](https://www.microsoft.com/en-us/ignite), where nearly 30.000 (you read correctly THIRTY THOUSAND) people gather together to see what's new for the various products and services Microsoft is offering. Me as a Conversational AI guy, I'm quite excited on the news and announcements they'll share at the event, but du to the fact, there the amount of news is that big, one has a hard time following all the necessary news and announcements at a glance. That's why I compose a blog post tackling the various areas of Conversational AI in order to provide a comprehensive list of news and further links, to look through in order to dive into certain things being announced. ## Bot Framework Here is the list of all news and announcements concering the Bot Framework: ### Bot Framework SDK * Bot Framework SDK v4.6 (GA) for [C#](https://github.com/Microsoft/botbuilder-dotnet/#packages) & [JS](https://github.com/Microsoft/botbuilder-js#packages) along with updated [docs](https://docs.microsoft.com/en-us/azure/bot-service?WT.mc_id=AI-MVP-5003073) and [samples](https://github.com/Microsoft/BotBuilder-Samples/tree/master/samples/csharp_dotnetcore) * [Bot Framework SDK for Microsoft Teams (GA)](https://docs.microsoft.com/azure/bot-service/bot-builder-basics-teams?WT.mc_id=AI-MVP-5003073) * [Bot Framework SDK for Skills (Preview)](https://microsoft.github.io/botframework-solutions/overview/skills/) * [Adaptive Dialogs (Preview)](https://github.com/Microsoft/BotBuilder-Samples/tree/master/experimental/adaptive-dialog/docs) * [Language Generation (Preview)](https://github.com/Microsoft/BotBuilder-Samples/tree/master/experimental/language-generation/docs) * [Common Expression Language (Preview)](https://github.com/Microsoft/BotBuilder-Samples/tree/master/experimental/language-generation/docs) * [Bot Framework for Power Virtual Agent (Preview)](https://docs.microsoft.com/en-us/dynamics365/ai/customer-service-virtual-agent/overview) * [LivePerson integration](https://developers.liveperson.com/third-party-bots-microsoft-bot-framework.html) * [New WeChat adapter](https://aka.ms/bf-wechat) ### Bot Framework Composer (Preview) So this is where I not only want to add a bullet list of links, but I actually want to share my thoughts on that briefly (and I will for sure follow-up with one or more posts on that solution). Bot Framework Composer is a game changer in my eyes in the field of creating/developing bots, as it gives you the ability to sketch out and compose the whole bot (along with the dialogs and all that good stuff) within a visual interface instead of writing actual code. And the cool thing about that is, when you are done modeling your dialogs and the whole conversation(s) you can actually use Composer to create the bot solution's code and run it immediately to test it in the emulator or deploy it to Azure using the CLI for example, still without writing a single line of code. So if you are building bots (no matter if you are a developer or IT Pro or citizen developer or business user or whatever other role you have) you should definetly check out this really awesome tool as this will be making your life even more easier in terms of building smart and engaging bots with ease. So if you want to learn more, just take a look at this [link](https://github.com/microsoft/BotFramework-Composer/blob/stable/README.md) and give it a go... {{< figure src="/images/054_composer.png" >}} ### Azure Bot Service * [Direct Line Speech (GA)](https://docs.microsoft.com/azure/bot-service/directline-speech-bot?WT.mc_id=AI-MVP-5003073) * [Direct Line App Service Extension (Preview)](https://aka.ms/directline-ase) ### Bot Framework Solutions * [Virtual Assistant Core](https://github.com/Microsoft/botframework-solutions#readme) * [Enterprise Assistant (Preview)](https://microsoft.github.io/botframework-solutions/solution-accelerators/assistants/enterprise-assistant/) ### Bot Framework Web Chat (GA) * [BF Webchat v4 went GA](https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-webchat-overview?WT.mc_id=AI-MVP-5003073) ### Bot Framework CLI (GA) * [New CLI released for managing Bot Framework bots and related services bundling the previously available standalone tools into one CLI](https://github.com/microsoft/botframework-cli) ## Cognitive Services Here is the list of all news and announcements concering the Cognitive Services: ### Language Understanding * [General availability of the prediction endpoint V3](https://docs.microsoft.com/en-gb/azure/cognitive-services/luis/luis-migration-api-v3?WT.mc_id=AI-MVP-5003073) * [Prebuilt domains is now generally available (GA)](https://docs.microsoft.com/en-gb/azure/cognitive-services/luis/luis-reference-prebuilt-domains?WT.mc_id=AI-MVP-5003073) * [Japanese prebuilt entities - age, currency, number, percentage are not support in V3](https://docs.microsoft.com/en-gb/azure/cognitive-services/luis/luis-reference-prebuilt-entities#japanese-entity-support?WT.mc_id=AI-MVP-5003073) * [Italian prebuilt entities - age, currency, dimension, number, percentage resolution changed from V2](https://docs.microsoft.com/en-gb/azure/cognitive-services/luis/luis-reference-prebuilt-entities#italian-entity-support?WT.mc_id=AI-MVP-5003073) * [Enhance user experience in preview.luis.ai portal - revamped labeling experience to enable building and debugging complex models](https://preview.luis.ai/) * [Advance language understanding capabilities](https://docs.microsoft.com/en-gb/azure/cognitive-services/luis/luis-concept-entity-types?WT.mc_id=AI-MVP-5003073) * [New and expanded limits](https://docs.microsoft.com/en-gb/azure/cognitive-services/luis/luis-boundaries?WT.mc_id=AI-MVP-5003073) ### QnA Maker * [Explicitly setting the language for all knowledge bases in the QnA Maker service](https://docs.microsoft.com/en-us/azure/cognitive-services/qnamaker/how-to/language-knowledge-base#select-language-when-creating-first-knowledge-base?WT.mc_id=AI-MVP-5003073) * [Multi-Turn feature (GA)](https://docs.microsoft.com/azure/cognitive-services/qnamaker/how-to/multiturn-conversation?WT.mc_id=AI-MVP-5003073) * [Batch testing support](https://aka.ms/qna_batchtest) ## Must-See On-Demand Sessions * **Azure AI** * [Invent with purpose on Azure](https://myignite.techcommunity.microsoft.com/sessions/87750?source=sessions) * [Azure AI: Powering new possibilities for every organization](https://myignite.techcommunity.microsoft.com/sessions/81277?source=sessions) * **Bot Framework** * [Three ways to build bots with Azure AI that can make you a hero in your organization](https://myignite.techcommunity.microsoft.com/sessions/83902?source=sessions) * [A deep-dive into conversational AI using Azure Bot Service and Cognitive Services](https://myignite.techcommunity.microsoft.com/sessions/81066?source=sessions) * [Create sophisticated, enterprise-ready bots from your existing data, with no code using Azure Cognitive Services QnA Maker](https://myignite.techcommunity.microsoft.com/sessions/83915?source=sessions) * [Let’s build a knowledgeable chatbot with Microsoft Bot Framework, Azure Bot Services, and QnA Maker](https://myignite.techcommunity.microsoft.com/sessions/79311?source=sessions) * [Creating enterprise conversational user experiences with virtual assistants](https://myignite.techcommunity.microsoft.com/sessions/81065?source=sessions) * **Cognitive Services** * [What’s new with Azure Cognitive Services](https://myignite.techcommunity.microsoft.com/sessions/89303?source=sessions) * [Building enterprise ready, scalable AI solutions using Azure Cognitive Services](https://myignite.techcommunity.microsoft.com/sessions/81067?source=sessions) * [Automating Azure Cognitive Services with PowerShell](https://myignite.techcommunity.microsoft.com/sessions/80777?source=sessions) * [AI behind call center transcription: Using Azure Cognitive Services to improve customer experience](https://myignite.techcommunity.microsoft.com/sessions/81075?source=sessions) * [AI behind call center analytics: Using Azure Cognitive Services to improve customer experiences](https://myignite.techcommunity.microsoft.com/sessions/81072?source=sessions) * [Integrate speech and text translation into your apps using Azure Cognitive Services Translator](https://myignite.techcommunity.microsoft.com/sessions/83916?source=sessions) * [Advanced Natural Language Understanding (NLU) models using LUIS and Azure Cognitive Services](https://myignite.techcommunity.microsoft.com/sessions/83914?source=sessions) * [How to implement automated and intelligent enterprise support using Azure Cognitive Services and Microsoft Dynamics 365](https://myignite.techcommunity.microsoft.com/sessions/84634?source=sessions) --- title: "Bot Framework v4 SPFX Web Part Released" url: https://bisser.io/bot-framework-v4-spfx-web-part-released/ markdown: https://bisser.io/bot-framework-v4-spfx-web-part-released/index.md date: 2019-10-03 lastmod: 2019-10-03 tags: AI, Bot Framework description: "Bot Framework v4 SPFX Web Part Released" status: legacy reading_time_min: 3 --- ## The Background So I had this idea already a couple of months to release or better open-source a [SharePoint Framework](https://docs.microsoft.com/en-us/sharepoint/dev/spfx/sharepoint-framework-overview) WebPart which is capable of bringing your [Bot Framework](https://dev.botframework.com) conversations (= your bot) to SharePoint (Online). Now I finally had the chance of publishing my webpart to the [official SPFx samples GitHub repo](https://github.com/SharePoint/sp-dev-fx-webparts/tree/master/samples/react-bot-framework) to make it available for everyone. So if you want to try it out and integrate it into your SharePoint tenant, just clone the repo, and follow the instructions mentioned [here](https://github.com/SharePoint/sp-dev-fx-webparts/tree/master/samples/react-bot-framework#minimal-path-to-awesome) and you should be able to get it up and running in a couple of minutes. ## The WebPart So as this is a SPFx webpart, you can integrate it quite easily on every modern page. After you've added it you need to configure the webpart using it's property pane: {{< figure src="/images/052_webpart_propertypane.png" >}} {{< figure src="/images/052_webpart_propertypane2.png" >}} ### Properties So the following properties are available to alter the look and feel of your BF4 webchat webpart: | Name | Description | | ----------------------------- | -------- | | Direct Line Secret | The direct line secret of your bot which you can get from the Azure Portal | | Background color of webchat | This is the background color of the complete webchat window as a Hex value | | Bot messages background color | Hex value for the bot messages background color | | Bot messages foreground color | Hex value for the bot messages foreground color | | User messages background color| Hex value for the user messages background color | | User messages foreground color| Hex value for the user messages foreground color | | Sendbox background color | Hex value for the sendbox background color | | Sendbox text color | Hex value for the sendbox text color | | Avatar image used for bot | This is the image which you can use as an icon next to each message coming from the bot | | Avatar initials used for bot | Instead of using an icon you can also use initials to be displayed next to the bot's messages | | Avatar image used for user | This is the image which you can use as an icon next to each message coming from the user (you could also enhance this and get the image from the Microsoft Graph) | | Avatar initials used for user | nstead of using an icon you can also use initials to be displayed next to the user's messages | | Disable upload button | If you like you can enable/disable to upload button to prevent users from uploading documents to the conversation | So if you add/modify all those values to fit your needs you should be able to have a custom webchat component according to your CI/CD guidelines right in your SharePoint environment. The most crucial part of that is the Direct Line Secret, as this is used to establish the connection between your webpart and your bot - so make sure you use the right one. {{< figure src="/images/052-bot-frameworkv4-webpart-preview.png" >}} And I also have an idea for the enhancement of that, to show this on every page in a sitecollection or even hub site, so stay tuned for more to come: {{< figure src="/images/052-bot-frameworkv4-webpart-preview-2.png" >}} Finally, if you wanto to enhance the webpart, it's open source, so just fork the repo and adapt it and the make a PR from [this repo here](https://github.com/SharePoint/sp-dev-fx-webparts/tree/master/samples/react-bot-framework)... Happy using! --- title: "Bot Framework v4 Webchat Styling Options" url: https://bisser.io/bot-framework-v4-webchat-styling-options/ markdown: https://bisser.io/bot-framework-v4-webchat-styling-options/index.md date: 2019-09-12 lastmod: 2019-09-12 tags: AI, Bot Framework description: "Bot Framework v4 Webchat Styling Options" status: legacy reading_time_min: 4 --- ## The Starting Position Nowadays, styling is a hot topic. *If it looks good - I'll use it* is often the mindset of users. So it's very important to offer something very good-looking to be successful with what you do. And so it is with bots. The [Microsoft Bot Framework team](https://github.com/microsoft/BotFramework-WebChat) did a pretty good job in styling the webchat component which is looking pretty good out of the box: ![Microsoft Bot Framework Webchat](https://raw.githubusercontent.com/microsoft/BotFramework-WebChat/master/media/weatherquery.png.jpg) But companies often have the need to apply their CI/CD guidelines on whatever they use. So the white & black look of the webchat component is often not exactly what they need. Therefore, the Microsoft Bot Framework v4 webchat is offering the possibility to be customized in terms of styling. This is a great way for devs to modify the look and feel of the webchat. In the [BFv4 Webchat GitHub repo](https://github.com/microsoft/BotFramework-WebChat/blob/master/SAMPLES.md) there is a very detailed documentation on how to style the webchat using CSS within the HTML code. But looking at the following table which was taken from [here](https://github.com/microsoft/BotFramework-WebChat) you can see, that the React component offers even more options in terms of customization: | | CDN bundle | React | | ----------------------------- | :--------: | :------: | | Change colors | ✔ | ✔ | | Change sizes | ✔ | ✔ | | Update/replace CSS styles | ✔ | ✔ | | Listen to events | ✔ | ✔ | | Interact with hosting webpage | ✔ | ✔ | | Custom render activities | | ✔ | | Custom render attachments | | ✔ | | Add new UI components | | ✔ | | Recompose the whole UI | | ✔ | So I was currently searching a while until I found the documenation on how to customize the webchat styling when using it as a React component, which I want to share. It's basically described [here](https://github.com/microsoft/BotFramework-WebChat/blob/master/packages/component/src/Styles/defaultStyleOptions.js). The following list shows the default values, which, of course you can adapt to fit your needs. ## StyleSetOptions The following samples show the default style set options which can be modified to style the React webchat component. ### Color and paddings The default colors for the webchat background and paddings: ```JavaScript accent: '#0063B1', backgroundColor: 'White', cardEmphasisBackgroundColor: '#F0F0F0', paddingRegular: 10, paddingWide: 10 * 2, subtle: '#767676' ``` ### Word break The defaults for word breaks: ```JavaScript messageActivityWordBreak: 'break-word', // 'normal' || 'break-all' || 'break-word' || 'keep-all' ``` ### Fonts The default fonts to be used: ```JavaScript fontSizeSmall: '80%', monospaceFont: fontFamily(['Consolas', 'Courier New', 'monospace']), primaryFont: fontFamily(['Calibri', 'Helvetica Neue', 'Arial', 'sans-serif']) ``` ### Avatar The default styling for the avatars of the user and the bot: ```JavaScript avatarSize: 40, botAvatarBackgroundColor: undefined, // defaults to accent color botAvatarImage: '', botAvatarInitials: '', userAvatarBackgroundColor: undefined, // defaults to accent color userAvatarImage: '', userAvatarInitials: '' ``` ### Bubble The default configuration for the bubbles, which is used to render chat messages (_NOTE: The bubble* are used for bot messages and the bubbleFromUser* are used for rendering user messages_): ```JavaScript bubbleBackground: 'White', bubbleBorderColor: '#E6E6E6', bubbleBorderRadius: 2, bubbleBorderStyle: 'solid', bubbleBorderWidth: 1, bubbleFromUserBackground: 'White', bubbleFromUserBorderColor: '#E6E6E6', bubbleFromUserBorderRadius: 2, bubbleFromUserBorderStyle: 'solid', bubbleFromUserBorderWidth: 1, bubbleFromUserNubOffset: 'bottom', bubbleFromUserNubSize: 0, bubbleFromUserTextColor: 'Black', bubbleImageHeight: 240, bubbleMaxWidth: 480, // screen width = 600px bubbleMinHeight: 40, bubbleMinWidth: 250, // min screen width = 300px, Edge requires 372px (https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/13621468/) bubbleNubOffset: 'bottom', bubbleNubSize: 0, bubbleTextColor: 'Black' ``` ### Markdown The default styling for markdown: ```JavaScript markdownRespectCRLF: true ``` ### Rich Cards The default styling for rich cards: ```JavaScript richCardWrapTitle: false, // Applies to subtitles as well ``` ### Root The default styling for the root properties: ```JavaScript rootHeight: '100%', rootWidth: '100%', ``` ### Scroll to end button The default setting for the scroll to end button to be visible: ```JavaScript hideScrollToEndButton: false, ``` ### Send box The default styling options for the send box: ```JavaScript hideSendBox: false, hideUploadButton: false, microphoneButtonColorOnDictate: '#F33', sendBoxBackground: 'White', sendBoxButtonColor: undefined, // defaults to subtle sendBoxButtonColorOnDisabled: '#CCC', sendBoxButtonColorOnFocus: '#333', sendBoxButtonColorOnHover: '#333', sendBoxDisabledTextColor: undefined, // defaults to subtle sendBoxHeight: 40, sendBoxMaxHeight: 200, sendBoxTextColor: 'Black', sendBoxBorderBottom: '', sendBoxBorderLeft: '', sendBoxBorderRight: '', sendBoxBorderTop: 'solid 1px #E6E6E6', sendBoxPlaceholderColor: undefined, // defaults to subtle sendBoxTextWrap: false, ``` ### Visually show spoken text Default option for spoken text to be visible: ```JavaScript showSpokenText: false, ``` ### Suggested actions Defaults for suggested action styling: ```JavaScript suggestedActionBackground: 'White', suggestedActionBorder: undefined, // split into 3, null suggestedActionBorderColor: undefined, // defaults to accent suggestedActionBorderStyle: 'solid', suggestedActionBoarderWidth: 2, suggestedActionBorderRadius: 0, suggestedActionImageHeight: 20, suggestedActionTextColor: null, suggestedActionDisabledBackground: undefined, // defaults to suggestedActionBackground suggestedActionDisabledBorder: null, suggestedActionDisabledBorderColor: '#E6E6E6', suggestedActionDisabledBorderStyle: 'solid', suggestedActionDisabledBorderWidth: 2, suggestedActionDisabledTextColor: undefined, // defaults to subtle suggestedActionHeight: 40, ``` ### Timestamp The default values for rendering timestamps: ```JavaScript timestampColor: undefined, // defaults to subtle timestampFormat: 'relative', // 'absolute' ``` ### Transcript overlay buttons Defaults for overlay buttons (e.g. carousel and suggested action flippers, scroll to bottom, etc.): ```JavaScript transcriptOverlayButtonBackground: 'rgba(0, 0, 0, .6)', transcriptOverlayButtonBackgroundOnFocus: 'rgba(0, 0, 0, .8)', transcriptOverlayButtonBackgroundOnHover: 'rgba(0, 0, 0, .8)', transcriptOverlayButtonColor: 'White', transcriptOverlayButtonColorOnFocus: undefined, // defaults to transcriptOverlayButtonColor transcriptOverlayButtonColorOnHover: undefined, // defaults to transcriptOverlayButtonColor ``` ### Video Default options for rendering videos as messages: ```JavaScript videoHeight: 270, // based on bubbleMaxWidth, 480 / 16 * 9 = 270 ``` ### Connectivity UI Default options for the connectivity UI rendering: ```JavaScript connectivityIconPadding: PADDING_REGULAR * 1.2, connectivityMarginLeftRight: PADDING_REGULAR * 1.4, connectivityMarginTopBottom: PADDING_REGULAR * 0.8, connectivityTextSize: '75%', failedConnectivity: '#C50F1F', slowConnectivity: '#EAA300', notificationText: '#5E5E5E', typingAnimationBackgroundImage: null, typingAnimationDuration: 5000, typingAnimationHeight: 20, typingAnimationWidth: 64, spinnerAnimationBackgroundImage: null, spinnerAnimationHeight: 16, spinnerAnimationWidth: 16, spinnerAnimationPaddingRight: 12, enableUploadThumbnail: true, uploadThumbnailContentType: 'image/jpeg', uploadThumbnailHeight: 360, uploadThumbnailQuality: 0.6, uploadThumbnailWidth: 720 ``` ## Conclusion From my point of view it's extremely important to use styling options to give your virtual assistant a more personal touch. That's why I recommend everyone to walk through the [docs](https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-webchat-customization?WT.mc_id=AI-MVP-5003073) which explain the process of customizing the BFv4 webchat component pretty good. Have fun customizing! --- title: "Using Custom Channel Data in Bot Framework v4" url: https://bisser.io/using-custom-channel-data-in-bot-framework-v4/ markdown: https://bisser.io/using-custom-channel-data-in-bot-framework-v4/index.md date: 2019-08-28 lastmod: 2019-08-28 tags: AI, Bot Framework description: "Using Custom Channel Data in Bot Framework v4" status: legacy reading_time_min: 4 --- ## Problem statement Imagine this: You are working on a Bot Framework bot and want to implement human [handoff](https://docs.microsoft.com/en-us/azure/bot-service/bot-service-design-pattern-handoff-human?WT.mc_id=AI-MVP-5003073) to give users the option to have a conversation with another human being if the bot is not able to answer the user's questions. Now there are many good [samples](https://github.com/OfficeDev/microsoft-teams-faqplusplus-app) out there for channels like Microsoft Teams for example which demonstrate how to bring in a human into the loop. But I was trying to find a way to tell the webchat control (via DirectLine) that a given point in the conversation it should do something (escalate to an agent). Now there was one thing I found which could do the trick for me: **Custom Channel Data Property** (you can find out what that is in detail [here](https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-channeldata?WT.mc_id=AI-MVP-5003073)). The article basically describes it as follows: > Some channels provide features that cannot be implemented by using only message text and attachments. To implement channel-specific functionality, you can pass native metadata to a channel in the activity object's channel data property. For example, your bot can use the channel data property to instruct Telegram to send a sticker or to instruct Office365 to send an email. As you can already see in that description, you can tell your bot's channel that it should do something for you. And that's the thing I needed. So I went for the docs to see on how to implement it in my .NET Echo Bot for testing purposes and found [this article](https://blog.botframework.com/2017/03/28/custom-channel-data/) by the [Microsoft Bot Framework Team](https://twitter.com/msbotframework) instructing on how to send such a custom channel data object (it was written for Facebook but I surely can use it for my webchat scenario as well I thought). But upon reading this article I recognized something was odd: *this article was written in March 2017 and used Bot Framework v3 instead of v4*. This means that I cannot re-use that code as I would like because the majority of the implementation has changed. So I had to look further and googled a couple of hours without any helpful results. My problem was that I needed a way to create a message and add the channel data property to that message before sending it to the channel. Normally, you would create a message like this in your .NET bot: ```C# await turnContext.SendActivityAsync(MessageFactory.Text($"Echo: {turnContext.Activity.Text}"), cancellationToken); ``` But the problem with this is that I have no option to add any properties to my *MessageFactory.Text* attribute here. Next thing I tried was to create a Json object which actually represented my channel data property and sent that via the exact same message as above to the webchat channel. And it got rendered as exactly that: a Json string containing my information which was provided to the user - not what I wanted, so I needed to look further. The docs also said nothing about the actual implementation in either .NET or JS the only thing they mention is how the format should look like. But this was not everything I needed to know... ## Solution And after researching many Google pages, I suddenly found [this issue](https://github.com/microsoft/botframework-sdk/issues/5061) on Github, which was exactly what I needed. I needed the following line as a starting point to get down the road I wanted to end: ```C# IMessageActivity message = Activity.CreateMessageActivity(); ``` Now I could finally add the custom channel data property to my message before sending it out to the webchat channel using the following lines: ```C# message.Text = $"Trying to connect to an agent"; message.TextFormat = "plain"; message.ChannelData = new Dictionary { ["type"] = "connectAgent", ["Data"] = new Dictionary { { "AgentSkill", "ChatBot" } } }; await turnContext.SendActivityAsync(message, cancellationToken); ``` And now that was what I needed. Now I could talk to my bot and whenever he is not able to give me an appropriate answer I can send the channel data property saying it should connect me to an agent: {{< figure src="/images/bfv4channeldata1.png" >}} ## Conclusion After many hours of research I finally found a way on how to add custom channel properties to messages before sending them to a BFv4 channel. The code above is not only working for the webchat channel but for any other channel, just stick to the channel's required format descibed [here](https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-channeldata?WT.mc_id=AI-MVP-5003073) and you should be able to send your channel custom metadata to trigger certain actions (for exampling sending an automated email after the user has left the conversation with the chat transcript attached for further storing or analytics purposes - which may be the topic of my next post 😉) --- title: "QnA Maker Postman Collection-Template now available" url: https://bisser.io/qna-maker-postman-collection-template-now-available/ markdown: https://bisser.io/qna-maker-postman-collection-template-now-available/index.md date: 2019-07-11 lastmod: 2019-07-11 tags: AI, Bot Framework description: "QnA Maker Postman Collection-Template now available" status: legacy reading_time_min: 3 --- As I teamed up with [Albert-Jan Schot](https://twitter.com/appieschot), [Rick Van Rousselt](https://twitter.com/RickVanRousselt) & [Thomas Gölles](https://twitter.com/thomyg) to create a project called [SelectedTech](https://www.youtube.com/channel/UC06iQ_SyfdKsveFmKwFtC4Q?view_as=subscriber), where we discuss and share our knowledge on Office 365/Teams/SharePoint & Bots and record that as webinars, we had the idea of doing something which would be a cool way of helping others with one of the [Cognitive Service APIs](https://azure.microsoft.com/en-us/services/cognitive-services/) - [QnA Maker](https://qnamaker.ai). The idea was to create a Postman collection which we'll share later on, to bundle all API endpoints for the QnA Maker to easily test and debug the API. The inspiration to this was coming from Jeremy Thake and his project on the [Microsoft Graph Postman collection](https://twitter.com/jthake/status/1145744139763322881) (Thanks for that Jeremy - great thing!) and we thought, why not doing a similar thing for QnA Maker as well since we are working with that in our webinar projects. So I sat down and created such a Postman collection which you can find in our [GitHub repo](https://github.com/selectedtech/Samples). So this blog post should walk you through on how to use that for your own QnA Maker projects to ease the development and debugging phase a bit. **So if you want to see that in action, just watch our video:** [![SelectedTech - QnA Maker Postman Collection](http://img.youtube.com/vi/MJ5suH3X0FY/0.jpg)](http://www.youtube.com/watch?v=MJ5suH3X0FY "SelectedTech - QnA Maker Postman Collection") ## Step 1 - Set up your QnA Maker service If you haven't done so already, you need to create your own QnA Maker instance. I won't walk you through that on my blog, because this is very well documented by Microsoft. So please follow the guide [here](https://docs.microsoft.com/en-us/azure/cognitive-services/qnamaker/quickstarts/create-publish-knowledge-base?WT.mc_id=AI-MVP-5003073) to create your service and knowledgebase. Now make sure to insert some QnA pairs to your KB so we can see if we get the correct results back by our Postman collection. ## Step 2 - Set up Postman Now there are actually 2 ways how you can use this Postman collection (choose either one of the following): ### Step 2.1 - Import Postman collection If you want to import the current status of the collection which might be updated frequently you'll need to do the following (those steps are also described [here](https://github.com/selectedtech/Samples/tree/master/QnAMakerPostmanCollection)): **1.** Download and register for [Postman](https://www.getpostman.com/). **2.** Click **File | Import ...**. **3.** Select **Import From Link**. **4.** Paste the following two URLs and click Import after each. `https://raw.githubusercontent.com/selectedtech/Samples/master/QnAMakerPostmanCollection/QnA%20Maker%20Postman%20Collection.postman_collection.json` `https://raw.githubusercontent.com/selectedtech/Samples/master/QnAMakerPostmanCollection/QnA%20Maker%20Environment.postman_environment.json` You should now see the **QnA Maker Postman Collection** collection on the left had side Collections pane. **5.** Click on the **No environment** drop down in top right hand corner. **6.** Select **QnA Maker Environment**. **7.** Click the **eye** icon to the the right and then click **Edit**. **8.** Enter in to the **current** (not **initial**) variables your values (e.g. Subscription Key, region, host, ...) Now you should be ready to go and hit your QnA Maker API endpoints! But there is another more convenient way of adding the QnA Maker Postman collection to your Postman app... ### Step 2.2 - Import QnA Maker Postman template Now if you want to go the more convenient way of adding this collection to your workspace, simply open the Postman app on your machine. Upon starting it, you should see a popup saying "Create New" on the top left side. Now switch to "Templates" and search for "QnA Maker" which should give you this screen: {{< figure src="/images/capture20190711110626541.png" >}} After clicking on it, you'll be asked if you want to "Use this template" (which you obviously want ;) ): {{< figure src="/images/capture20190711110720137.png" >}} Now after the collection has been imported, you'll need to add all of the variables (keys, IDs, ...) to your "QnA Maker Environment" (top right corner in the Postman client) and you should be good to go to hit the API endpoint now. * * * Disclaimer: This is an open source project which everyone is allowed to contribute to. Feel free to open PRs on the [GitHub repo](https://github.com/selectedtech/Samples/) and we would be happy to get more functionality into this anytime soon 😉 --- title: "Microsoft Build 2019 updates on Conversational AI" url: https://bisser.io/microsoft-build-2019-updates-on-conversational-ai/ markdown: https://bisser.io/microsoft-build-2019-updates-on-conversational-ai/index.md date: 2019-05-09 lastmod: 2019-05-09 tags: AI, Bot Framework, Cognitive Services description: "Microsoft Build 2019 updates on Conversational AI" status: legacy reading_time_min: 4 --- As every year, Microsoft is hosting it's Build conference again in Seattle aiming at showing the world all news around the latest developer tools and tech. As there is a ton of news, as usual, this blog post should give you an overview on the news and infos on the Conversational AI announcements at Build: ## Bot Framework * [Adaptive Dialog](https://github.com/Microsoft/BotBuilder-Samples/tree/master/experimental/adaptive-dialog) | [docs](https://github.com/Microsoft/BotBuilder-Samples/tree/master/experimental/adaptive-dialog/docs) | [C# samples](https://github.com/Microsoft/BotBuilder-Samples/tree/master/experimental/adaptive-dialog/csharp_dotnetcore) :: Simplify the development of sophisticated dialogs by dynamically updating as the conversation unfolds. This is especially useful when the context of conversation is switched or if there are interruptions in the middle of a conversation. * [Language Generation](https://github.com/Microsoft/BotBuilder-Samples/tree/master/experimental/language-generation) | [docs](https://github.com/Microsoft/BotBuilder-Samples/tree/master/experimental/language-generation/docs) | [C# samples](https://github.com/Microsoft/BotBuilder-Samples/tree/master/experimental/language-generation/csharp_dotnetcore) :: Streamlines the creation of smart and dynamic bot responses by constructing meaningful, variable and grammatically correct responses that a bot can send back to the user. * [Common Expression Language](https://github.com/Microsoft/BotBuilder-Samples/tree/master/experimental/common-expression-language) | [api](https://github.com/Microsoft/BotBuilder-Samples/blob/master/experimental/common-expression-language/api-reference.md) :: Both Adaptive dialogs and Language Generation rely on and use a common expression language to power bot conversations. * [Botkit](https://github.com/howdyai/botkit#readme) :: Major version update with support for Bot Framework Emulator, Azure Bot Service. New model for handling interruptions and composable dialogs. * [Bot Framework Channels](https://github.com/microsoft/botframework-sdk#channels-and-adapters) * Enhanced Speech capabilities, Direct Line Speech Channel – (private preview) Streamlined integration of the Bot Framework and Microsoft's Speech Services in [Azure Bot Service](https://github.com/microsoft/botframework/blob/master/README.md#ABS-whats-new) as a new channel, with support for custom wake words and bi-directional speech and text. To sign up, add the Direct Line Speech channel to your Azure Bot Service. * SDK Channel adapters available [here](https://github.com/howdyai/botkit#readme) - New platform adapters for Web chat, Slack, Webex, Facebook, Google Hangouts and Twilio SMS. Work with Bot Framework V4 SDK (JS only). Additional new adapters put together by the [BotBuilderCommunity](https://github.com/BotBuilderCommunity/botbuilder-community-dotnet/tree/develop/libraries) * [Bot Framework Skills](https://github.com/Microsoft/AI/tree/master/docs#readme) * Skills include language models, dialogs, and cards that can be reusable across bots. Examples include productivity (calendar, email and task productivity- powered by Microsoft Graph), industry (point of interest, and automotive), and custom. * [Virtual Assistant Template](https://github.com/Microsoft/AI/tree/master/docs#readme) * This template incorporates the previous Enterprise Template and brings together all of the best practices and supporting components identified through building advanced assistant-like conversational experiences. It greatly simplifies the creation of a new bot project including: basic conversational intents, Dispatch integration, QnA Maker, Application Insights and an automated deployment. * [Bot Framework Emulator](https://github.com/microsoft/botframework/blob/master/README.md#Emulator-whats-new) * Bot Inspector, a new ability to debug live conversations in your bots on any channel or adapter - Microsoft Teams, Slack, Cortana, Facebook Messenger, Skype, etc. ## Language Understanding (LUIS) * [Preview of V3 API migration guide](https://docs.microsoft.com/en-us/azure/cognitive-services/luis/luis-migration-api-v3?WT.mc_id=AI-MVP-5003073) * The query prediction endpoint APIs have changed. Use this guide to understand how to migrate to version 3 endpoint APIs. This V3 API provides the following new features, which include significant JSON request and/or response changes: * [External entities](https://docs.microsoft.com/en-us/azure/cognitive-services/luis/luis-migration-api-v3#external-entities-passed-in-at-prediction-time?WT.mc_id=AI-MVP-5003073) * [Dynamic lists](https://docs.microsoft.com/en-us/azure/cognitive-services/luis/luis-migration-api-v3#dynamic-lists-passed-in-at-prediction-time?WT.mc_id=AI-MVP-5003073) * [Prebuilt entity JSON changes](https://docs.microsoft.com/en-us/azure/cognitive-services/luis/luis-migration-api-v3#prebuilt-entities-with-new-json?WT.mc_id=AI-MVP-5003073) * [Improved analytics dashboard](https://docs.microsoft.com/en-us/azure/cognitive-services/luis/luis-how-to-use-dashboard?WT.mc_id=AI-MVP-5003073) * Find and fix problems with your trained app's intents when you are using example utterances. The summary dashboard displays overall app information, with highlights of intents that should be fixed. Review Dashboard analysis is an iterative process, repeated as you change and improve your model. * [Improved prebuilt domains](https://docs.microsoft.com/en-us/azure/cognitive-services/luis/luis-reference-prebuilt-domains?WT.mc_id=AI-MVP-5003073) * This reference provides information about the [prebuilt domains](https://docs.microsoft.com/en-us/azure/cognitive-services/luis/luis-how-to-use-prebuilt-domains?WT.mc_id=AI-MVP-5003073), which are prebuilt collections of intents and entities that LUIS offers. ## QnAMaker * [QnAMaker updates](https://github.com/microsoft/botframework/blob/master/README.md#QnA-whats-new) * **New! Extraction pipeline**: Now you can extract hierarchical information from URLs, files and SharePoint * **New! Intelligence**: Contextual ranking models, active learning suggestions * **New! Conversation**: Multi-turn conversations in QnA Maker * See this [sample](https://github.com/Microsoft/BotBuilder-Samples/tree/master/experimental/qnamaker-prompting) to create a multi-turn QnA Maker bot ## Cognitive Services * [A deep dive into what's new with Cognitive Services](https://azure.microsoft.com/en-us/blog/a-deep-dive-into-what-s-new-with-azure-cognitive-services/) If you want to check out the sessions related to the Conversational AI platform, here is a quick reference which sessions may be relevant: * [Vision Keynote](https://mybuild.techcommunity.microsoft.com/sessions/77571?source=sessions#top-anchor) by Satya Nadella, Microsoft CEO. * [Microsoft Azure](https://mybuild.techcommunity.microsoft.com/sessions/77572?source=sessions#top-anchor) by Scott Guthrie, EVP, CLOUD & ENTERPRISE * [The Microsoft Office 365 Platform](https://mybuild.techcommunity.microsoft.com/sessions/77573?source=sessions#top-anchor) by Rajesh Jha, EVP, EXPERIENCES + DEVICES * [How to use Azure Conversational AI to scale your business for the next generation - A deep dive into La Liga’s story ](https://mybuild.techcommunity.microsoft.com/sessions/76965?source=sessions#top-anchor)by Tulasi Menon, Nayer Wanas, Jesus Serrano Castro * [Your App, Your Device, Your Assistant - How to build a branded, voice-first Virtual Assistant](https://mybuild.techcommunity.microsoft.com/sessions/76969?source=sessions#top-anchor) by Darren Jefford, Travis Wilson, Khuram Shahid. * [Microsoft Research APIs – Project Personality Chat in action](https://mybuild.techcommunity.microsoft.com/sessions/77258?source=sessions#top-anchor), by Tulasi Menon * [Designing for Speech](https://mybuild.techcommunity.microsoft.com/sessions/77154?source=sessions#top-anchor) by Jessica Engström {{< figure src="/images/BotFrameworkDiagram.png" >}} --- title: "Setting up a Build and Release Pipeline for Azure Bot Service using Azure DevOps" url: https://bisser.io/setting-up-a-build-and-release-pipeline-for-azure-bot-service-using-azure-devops/ markdown: https://bisser.io/setting-up-a-build-and-release-pipeline-for-azure-bot-service-using-azure-devops/index.md date: 2019-02-07 lastmod: 2019-02-07 tags: AI, Bot Framework description: "Setting up a Build and Release Pipeline for Azure Bot Service using Azure DevOps" status: legacy reading_time_min: 8 --- This post is something I wanted to do for a long time already. In my opinion, the topic of CI/CD can be very beneficial in terms of easier deployments and delivery of solutions. That is why I decided to run through the process of a creating a CI/CD pipeline for an Azure Bot Service based on [Azure DevOps](https://azure.microsoft.com/en-us/services/devops/) which I want to share with you with this post. ## Preparation First of all, we need to prepare a couple of things before we can actually start building our CI/CD pipeline: ### Create Azure DevOps repository First of all, we need to create an Azure DevOps repository where we actually store our code. So head over to your Azure DevOps environment, which will be something like https://{tenantName}.visualstudio.com and create a new repository. {{< figure src="/images/01_stephan_bisser_devops_bot-1024x555.png" >}} Then we use the command line to clone the repo to our local disk and initialize the Git repo locally. To do that open your CMD of choice, navigate to a folder where you would like to store the source code and execute the command ```shell git clone https://{tenantName}.visualstudio.com/{projectName/_git/{repoName} ``` {{< figure src="/images/02_stephan_bisser_devops_bot-1024x555.png" >}} After the Git repo is initialized locally, we can now create a new VS bot project in that folder which will allow us to commit and push the code we are building locally to Azure DevOps for further processing. This way, we can develop and test our solution locally and if we say the code shall be deployed to Azure, we can just commit and push and the pipeline will do the rest, as we will see later on. {{< figure src="/images/03-1_stephan_bisser_devops_bot.png" >}} Make sure to create the project in the root folder where you cloned your repo into. {{< figure src="/images/03-2_stephan_bisser_devops_bot.png" >}} After the solution has been created, we can give it a go and test it locally, to see if the solution is building correctly. So hit F5 in Visual Studio and open the .bot file which has been created with the [Bot Framework Emulator](https://github.com/Microsoft/BotFramework-Emulator) and type in a message. {{< figure src="/images/04_stephan_bisser_devops_bot-1024x555.png" >}} So now that we have our repo stuff ready, we can go on and develop the bot logic locally. If we then say we are ready for production we can commit and push the changes to our Azure DevOps repo with the following 3 commands: ```shell git add . git commit -a -m "First commit" git push origin master ``` {{< figure src="/images/05_stephan_bisser_devops_bot-790x1024.png" >}} Now that the code has been committed and pushed we can check if the files have been created in Azure DevOps. {{< figure src="/images/06_stephan_bisser_devops_bot-1024x555.png" >}} ## Create a Build Pipeline After having our code in the Azure DevOps repo, we can set up our build pipeline to build that code in the repo automatically as well. So within your DevOps tenant navigate to pipelines and add a new Build pipeline. You'll need to select a template or start with an empty job (I'd go with a template as it saves time). In our case, it makes sense to select the "Azure Web App for ASP.NET template as we are building a C# bot. {{< figure src="/images/07-1_stephan_bisser_devops_bot-1024x555.png" >}} Now we would normally need to configure some settings for our build pipeline like the Azure Subscription we want to deploy the code to. But in our case we want to separate the build pipeline from the release pipeline to avoid any troubles afterward, so we delete the task "Azure App Service Deploy" from our task list completely. {{< figure src="/images/07-2_stephan_bisser_devops_bot-1024x555.png" >}} {{< figure src="/images/07-3_stephan_bisser_devops_bot-1024x555.png" >}} After that, we can save and queue our pipeline to make it ready to build our code. {{< figure src="/images/07-4_stephan_bisser_devops_bot-1024x555.png" >}} ## Create a Release Pipeline Now that we have our build pipeline ready, we need to create a release pipeline as well in order to actually deploy our built code to an Azure App Service. To do that head over to "Pipelines" and create a new release pipeline. Similar to the build pipeline we will go for the "Azure App Service deployment" template there. {{< figure src="/images/08-1_stephan_bisser_devops_bot-1024x555.png" >}} After creating, the pipeline will look pretty empty, but we will change that in a second. {{< figure src="/images/08-2_stephan_bisser_devops_bot-1024x555.png" >}} First of all, we need to add an [artifact](https://docs.microsoft.com/en-us/azure/devops/pipelines/release/artifacts?view=azure-devops) to the pipeline, which actually the finished "product" from our build pipeline. So select your project, source (which is our build pipeline) and the version you want to add to the pipeline (in my case I will go for the latest version). {{< figure src="/images/08-3_stephan_bisser_devops_bot-1024x555.png" >}} Now we added the artifact to the pipeline, we need to add some tasks to our stage for deploying the bot. But we will also create the necessary Azure resources within our pipeline in order to automate that task as well. So we need to add 2 tasks for executing Azure CLI commands to our stage before the task "Azure App Service Deploy" will be executed. Make sure to select your  Azure subscription and select "Inline script" as a script location. Then paste in the following command and hit save: ```shell az group create --location {location} --name {resourceGroupName} ``` {{< figure src="/images/08-4_stephan_bisser_devops_bot-1024x555.png" >}} The second Azure CLI command will create an Azure App Service Plan and an App Service. So we do the same thing as with the first task but the command should look like this: ```shell call az appservice plan create --name {appServicePlan} --resource-group {resourceGroupName} --sku FREE call az webapp create --name {appServiceName} --resource-group {resourceGroupName} --plan {appServicePlan} ``` {{< figure src="/images/08-5_stephan_bisser_devops_bot-1024x555.png" >}} Then we need to configure the 3rd task where we need to select the Azure subscription again and insert our App Service name from the previous task. {{< figure src="/images/08-6_stephan_bisser_devops_bot-1024x555.png" >}} Now the last thing to do here is to enable the continuous deployment trigger to make sure that the releases pipeline will run each time a new build is available. {{< figure src="/images/08-7_stephan_bisser_devops_bot-1024x555.png" >}} Now that we have that we will save it and check it again. The pipeline should look something like this: {{< figure src="/images/08-8_stephan_bisser_devops_bot-1024x555.png" >}} ## Trigger the first Build and Release Now that everything is in place, we can head over to our project in VS and commit and push again to trigger the build pipeline. {{< figure src="/images/09_stephan_bisser_devops_bot.png" >}} After the commit and push happened we should soon see that there is a new build available in Azure DevOps and if everything is correctly configured the output should be successful. {{< figure src="/images/10-1_stephan_bisser_devops_bot-1024x555.png" >}} Now that the build is available, the release pipeline should be triggered to create our resources in Azure and deploy the code to those resources. {{< figure src="/images/10-2_stephan_bisser_devops_bot-1024x555.png" >}} After you see all green lights, open the Azure Portal and check if the resources have been created correctly. {{< figure src="/images/11_stephan_bisser_devops_bot-1024x555.png" >}} ## Create a new Bot Channel Registration By the time of writing, I still have not quite figured out how to automate that task as well. The problem we have is that we need to create a new bot channel registration, BUT we need to create App ID and App Secret ourselves as we need to secret in our .bot file later on. So going with the automated routine using the CLI is no option at the moment as there is no way to grab that secret currently (I hope that there will be a solution to that soon). So within our Azure Portal we go to the App Service which has been created and grab the URL. {{< figure src="/images/12-1_stephan_bisser_devops_bot-1024x555.png" >}} Then we create a new service called ["Bot Channel Registration"](https://docs.microsoft.com/en-us/azure/bot-service/bot-service-quickstart-registration?WT.mc_id=AI-MVP-5003073) and select "Create App ID in the App Registration Portal": {{< figure src="/images/12-2_stephan_bisser_devops_bot.png-1024x555.png" >}} Now we generate a new app password (= appSecret) {{< figure src="/images/12-3_stephan_bisser_devops_bot.png-1024x555.png" >}} And after pasting that in into the Azure Portal we can create the service {{< figure src="/images/12-4_stephan_bisser_devops_bot.png-1024x555.png" >}} Now that the service has been created, we need to adapt our .bot file locally with that information. So we create a new endpoint there (make sure to append **"/api/messages"** to your App Service URL you copied before as an endpoint): ```JSON {             "type": "endpoint",             "appId": "{appID}",             "appPassword": "{appSecret}",             "endpoint": "https://{appServiceName}.azurewebsites.net/api/messages",             "id": "{someRandomID}",             "name": "production" } ``` {{< figure src="/images/13-1_stephan_bisser_devops_bot-1024x555.png" >}} ## Update the pipeline Now you might already know, that with the current configuration of our release pipeline, we have a problem. Because we have tasks to create a new App Service each time a new build is available. That won't work, as we already have created those services in Azure. So we need to just update the code for that service instead of creating a new one. The simplest way to do that is to create a new stage in the release pipeline (**NOTE**: I am currently figuring out how to work with conditions in the Azure CLI task to check if the service has already been created and if so skip the task and go straight to the code deployment). So we are going to clone the stage we created earlier. {{< figure src="/images/14-1_stephan_bisser_devops_bot-1024x555.png" >}} But this time we only will have one task, which is responsible for deploying the code there, instead of creating the Azure resources. Also make sure to add an App Setting to that task, which will deal with the locked files: **MSDEPLOY_RENAME_LOCKED_FILES = 1** {{< figure src="/images/14-2_stephan_bisser_devops_bot-1024x555.png" >}} Now that we have that, we will trigger the pipeline again by committing changes to our repo via the command line. {{< figure src="/images/14-3_stephan_bisser_devops_bot.png" >}} And if everything has run successfully, we can go ahead and test the bot in the Azure Web Chat emulator to see if the changes have been applied to the bot's code. {{< figure src="/images/15_stephan_bisser_devops_bot.png-1024x555.png" >}} Now you can go ahead and implement your bot's logic, add Cognitive Services like LUIS and QnAMaker to make your bot intelligent and whenever you are finished, just commit and push and your Azure resources will be updated by Azure DevOps automatically... ## Summary To sum up, the use of Azure DevOps as a CI/CD pipeline can help you in terms of developing locally, but building and deploying remotely. Thus, there are still some minor issues, like the problem, that we need to create the Azure Bot Channel Registration manually, due to the fact, that we cannot grab the App Password using the Azure CLI. But apart from that, we have a pretty cool way of integrating a CI/CD pipeline for building bots as well. **NOTE**: I will look into the Azure CLI task for creating the Azure resources and update this post when I have found a good way of handling conditions in that task to overcome the fact that we need to create a second stage for all deployments apart from the first one. So stay tuned… In case you want to see the actual implementation just take a look at [![SelectedTech - CI/CD with your bot and Azure DevOps](http://img.youtube.com/vi/6Wm1mK_KBAk/0.jpg)](http://www.youtube.com/watch?v=6Wm1mK_KBAk "SelectedTech - CI/CD with your bot and Azure DevOps") --- title: "Launching the Periodic Table of Azure Cognitive Services" url: https://bisser.io/launching-the-periodic-table-of-azure-cognitive-services/ markdown: https://bisser.io/launching-the-periodic-table-of-azure-cognitive-services/index.md date: 2019-01-02 lastmod: 2019-01-02 tags: AI, Bot Framework categories: AI, Cognitive Services description: "Launching the Periodic Table of Azure Cognitive Services" status: legacy reading_time_min: 1 --- I'm super excited to announce something I have been building for a couple of months now. I teamed up with [Matt Wade](https://twitter.com/thatmattwade) and [Niels Gregers Johansen](https://twitter.com/niegrejoh) who are the initiators of the [Periodic Table of Office 365](https://app.jumpto365.com/) to build the **[Periodic Table of Azure Cognitive Services](https://ai.jumpto365.com/)**. This table should help to understand which Cognitive Services are currently out there, including the following: * Categorization * Preview status indicators * Links to the product pages As this is the first version for this kind of table, there will be more features in later releases, including description pages. {{< figure src="/images/CogServicesTitle.png" >}} I'm super stoked to be a part of this awesome initiative and am therefore glad to get your feedback on this first version of the table. If you have any questions or ideas, which we could include in a later version, please feel free to reach out to me on [Twitter](https://twitter.com/stephanbisser), [LinkedIn](https://www.linkedin.com/in/stephan-bisser/) or via [Mail](mailto:stephan@bisser.at). --- title: "End-to-End Bot Development Workflow Infographic" url: https://bisser.io/end-to-end-bot-development-workflow-infographic/ markdown: https://bisser.io/end-to-end-bot-development-workflow-infographic/index.md date: 2018-11-06 lastmod: 2018-11-06 tags: AI, Bot Framework categories: AI, Bot Framework description: "End-to-End Bot Development Workflow Infographic" status: legacy reading_time_min: 2 --- Developing bots can be a lot of work for those of you, who may not know which tools Microsoft offers for the various bot development stages. Fortunately, Microsoft released an [article](https://github.com/Microsoft/botbuilder-tools/blob/master/tools-overview.md) which contains the various tools for the following phases in your end-to-end bot development workflow. The following phases need to be gone through combined with the correct toolset supporting your successful bot development project: * **Plan** * Plan your bot's conversations and dialogs * Take a look at the [principles of bot design](https://docs.microsoft.com/en-us/azure/bot-service/bot-service-design-principles?WT.mc_id=AI-MVP-5003073) * Use [Chatdown](https://github.com/Microsoft/botbuilder-tools/blob/master/tools-overview.md#create-mock-conversations-using-chatdown) to mock conversations before you actually start coding * **Build** * Use [Ludown](https://github.com/Microsoft/botbuilder-tools/blob/master/tools-overview.md#bootstrap-language-understanding-with-ludown) to bootstrap Language Understanding * Use [MSBot](https://github.com/Microsoft/botbuilder-tools/blob/master/tools-overview.md#keep-track-of-service-references-using-bot-file) to keep track of service references * Use [LUIS CLI](https://github.com/Microsoft/botbuilder-tools/blob/master/tools-overview.md#create-and-manage-luis-applications-using-luis-cli) to create and manage LUIS applications * Use [QnA Maker CLI](https://github.com/Microsoft/botbuilder-tools/blob/master/tools-overview.md#create-qna-maker-kb-using-qna-maker-cli) to create QnA Maker knowledgebases * Use [Dispatch CLI](https://github.com/Microsoft/botbuilder-tools/blob/master/tools-overview.md#create-dipsatch-model-using-dispatch-cli) to create your own dipsatch models * **Test** * Use the [Bot Framework Emulator V4](https://github.com/Microsoft/BotFramework-Emulator/releases) for testing your bot * **Publish** * Use the [Azure CLI](https://github.com/Microsoft/botbuilder-tools/tree/master/AzureCli) to create, download and publish your locally built bots to the Azure Bot Service * **Connect** * Use the [Azure CLI](https://github.com/Microsoft/botbuilder-tools/tree/master/AzureCli) to connect your bot to the various Azure Bot Service channels * **Evaluate** * Use the [Azure Portal](https://portal.azure.com) to improve the bot's capabilities and performance * Use [Application Insights](https://docs.microsoft.com/en-us/azure/bot-service/bot-service-manage-analytics?WT.mc_id=AI-MVP-5003073) to gain conversation-level reporting on user, message and channel data {{< figure src="/images/Cloudguy_End-to-End_Bot_Development_Workflow.webp" >}} The full image can be downloaded here: [End-to-End Bot Development Workflow](https://cloudguy.blob.core.windows.net/resources/infographics/Cloudguy_End-to-End_Bot_Development_Workflow.webp) Feel free to use it for internal use in your presentations or documents. The only point I ask you to do is to credit me and link this article to your documents where you use it. Please do not sell this as your own 😉 --- title: "Key Parts of an Enterprise Bot Design" url: https://bisser.io/key-parts-of-an-enterprise-bot-design/ markdown: https://bisser.io/key-parts-of-an-enterprise-bot-design/index.md date: 2018-10-20 lastmod: 2018-10-20 tags: AI, Bot Framework categories: AI, Bot Framework description: "Key Parts of an Enterprise Bot Design" status: legacy reading_time_min: 6 --- In the modern AI-enabled world, digital agents or bots (aka chatbots) are an extremely hot topic, which every company no matter of their industry will at least think about at the moment. Many big software vendors like Microsoft, Amazon, Google, you name it are offering their own set of tools and services which developers and architects can use in order to build such a conversational AI interface (aka chatbot). But in this modern world, it is always about being fast and have a short time to market, which often results in releases which are not quite ready. And many AI devs develop bots as digital agents, but forget some really essential key parts, which actually make such a digital agent great and better compared to others. As I often get asked, what the main parts of a great chatbot are and I really spent a lot of time thinking about a good anatomy of a bot, I decided to sketch up the key parts which I think are essential when talking about an "Enterprise Bot Design" and which I want to explain in some words here as well, why they are important. {{< figure src="/images/SB_Cloudguy_Chatbot_Key_Parts.png" >}} ## Rich Attachments In order to design a more intuitive and common user experience, your bot should be able to receive and send rich attachments like pictures, videos or even emojis and interpret them. This makes the conversation more human-aligned as many users will communicate with the bot as they do with colleagues or friends. Therefore, the bot should be able to handle different input types apart from text. ## Cards Cards are a great way to let your users interact with your chatbot in an interactive and time-saving way as cards offer you the flexibility to give your users predefined input types like buttons combined with images and other UI elements. This will definitively raise your user adoption as users will be able to respond quicker and more intuitive. ## Dialogs Dialogs are the heart & soul of your bot's conversation design. With dialogs, you can easily design the conversation for various bot use cases. If you want to have your bot being aware of multiple use cases, you can, of course, attach multiple dialogs to your bot's code and furthermore, you can stack dialogs and design dialog trees with parent-child structure. ## State In order to give your bot some sort of memory, you can make use of a state service, which allows you to store both user and conversation states. This will help your bot to remember users and the conversations the bot had with the users in order to resume conversations and dialogs at a given point in time later as well. ## Language Understanding Detecting the user's sentiment and act based on various intents is an essential skill almost every bot needs to have. With a service providing language understanding attached to your bot, you can make your "intelligent" and really understand what your users want to tell your bot. This also allows you to fire different dialogs within your conversation based on the user's intent. ## QnA One of the easiest and fastest ways to actually check your bot's use case is to set up a QnA bot which is able to respond to questions stored in a knowledge base. Setting up such a QnA aware bot is not rocket science and it will actually help you verify if your users will use the bot or not. In addition to that, this is always an extremely helpful use case for employee onboarding or general HR topics. ## Personality Before implementing your bot you need to think about the way your bot should behave. Should the bot be a more formal bot or should it be some sort of "new kid on the block" with an attitude? Make sure to align your bot to your corporate communication strategy in order to avoid distraction. Furthermore, be aware that you should also inform your users beforehand that they are actually talking to a machine and not to a human otherwise user adoption could decrease massively. ## Tap Working with buttons and other quick response actions make it easier your users interacting with the bot. If your users have the ability to tap on buttons instead of typing the response the communication will be faster and more mobile-friendly. ## Enterprise Social Give your users the ability to like or dislike responses as they are used to it from other social media platforms where they express their opinion to responses that way. Be sure that your bot will also react based on that feedback the users give the bot. ## Visual Feedback Especially if your bot's response takes some time because you need to query other services, for example, you need to give your users visual feedback to let them know that your bot is currently busy with responding to their input. One possible solution could be to add typing indicators to give your users a visual feedback and let them know your bot is responding. ## Channels Choose your bot's communication channels during your design phase in order to check if there are some differences in the way you need to develop your bot. With the Microsoft Bot Framework, it is even possible to write your bot's logic once and deploy it to various different channels like Microsoft Teams, Facebook Messenger or Slack at the same time. ## Mobile In order to give your users the chance to talk/write to your bot no matter of the device, they are using you need to deploy your bot to mobile-enabled channels as well. Hence, it is important to add quick response capabilities to make avoid long typing responses for mobile users. ## Integration Depending on your use case it could be beneficial that your bot has integrations into other services and systems like LOB apps in order to gather data from those systems and enrich your conversation as well as add more capabilities to your bot. ## Single Sign On If you integrate other services into your bot it is essential that you enable Single Sign On for your users to avoid multiple logins within your bot's conversation. This will increase user adoption too. ## Automate Routines One of the things a bot should be good at is to take over repetitive tasks from your users to give them the ability to focus on the business-critical work or have more time to do innovative stuff. ## Search If you want to give your users the chance to ask your bot for help and let your bot search for users, content or skills this could be one essential feature for your bot which your users will use. Summing up, this infographic should help you showcase which parts are important and should be included when you are in the phase of designing your enterprise chatbot. Of course, it is not always 100% correct as different use cases require different pieces which make up your bot's design, but it should give you a starting point when you actually do not quite know what parts to include in your chatbot. --- title: "Microsoft Bot Framework v4 Infographic [UPDATED]" url: https://bisser.io/microsoft-bot-framework-v4-infographic-updated/ markdown: https://bisser.io/microsoft-bot-framework-v4-infographic-updated/index.md date: 2018-10-16 lastmod: 2018-10-16 tags: AI, Bot Framework categories: AI, Bot Framework description: "Microsoft Bot Framework v4 Infographic [UPDATED]" status: legacy reading_time_min: 1 --- Almost a year ago [I published one of my first infographics](https://bisser.io/bot-framework-101-infographic/) which should visualize the various components of the Microsoft Bot Framework. Back then, the [Bot Framework](https://dev.botframework.com) was available in version 3.0 and as you can imagine, since then there have been a lot of updates and changes. Some weeks ago, Microsoft announced that version 4.0 is now GA with many improvements and enhancements. Therefore, I decided to update my version of the infographic in order to visualize what changed and what's important when talking about the Bot Framework, which I also renamed the "Microsoft Conversational AI Landscape": {{< figure src="/images/Cloudguy_Bot_Framework_v4_Infographic.webp" >}} The full image can be downloaded here: [Microsoft Conversational AI Landscape](https://cloudguy.blob.core.windows.net/resources/infographics/Cloudguy_Bot_Framework_v4_Infographic.webp) Feel free to use it for internal use in your presentations or documents. The only point I ask you to do is to credit me and link this article to your documents where you use it. Please do not sell this as your own 😉 --- title: "MSIgnite updates on conversational AI at a glance" url: https://bisser.io/msignite-updates-on-conversational-ai-at-a-glance/ markdown: https://bisser.io/msignite-updates-on-conversational-ai-at-a-glance/index.md date: 2018-09-25 lastmod: 2018-09-25 tags: AI, Bot Framework categories: AI, Bot Framework description: "MSIgnite updates on conversational AI at a glance" status: legacy reading_time_min: 1 --- Every year somewhere at the end of September Microsoft's biggest conference [Ignite](https://www.microsoft.com/en-us/ignite) attracts thousands of people with a massive amount of news and big announcements. To keep up with all these awesome news is nearly impossible, as there is so much to explore in this very short period of time. Therefore, I try to sum up and link to all big announcements and news Microsoft made during the Ignite 2018 on the topic of conversational AI... ## Bot Framework * [Microsoft Bot Builder v4 SDK hits GA status](https://azure.microsoft.com/en-us/blog/build-conversational-experiences-with-microsoft-bot-framework/) * [New Azure Bot Service template "Virtual Assistant" introduced](https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-virtual-assistant-introduction?WT.mc_id=AI-MVP-5003073) * [New Azure Bot Service template "Enterprise Bot" introduced](https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-enterprise-template-overview?WT.mc_id=AI-MVP-5003073) * [More than 60 Bot Builder samples published for Bot Builder v4 SDK in .NET, JS](https://github.com/Microsoft/botbuilder-samples) * [Updated documentation on the Azure Bot Service & MS Bot Framework for v4 SDK](https://docs.microsoft.com/en-us/azure/bot-service/bot-service-quickstart?WT.mc_id=AI-MVP-5003073) * [Bot Framework Web Chat control fully customizable](https://github.com/Microsoft/BotFramework-WebChat/blob/preview/README.md) ## Cognitive Services * [QnAMaker supports Chit-Chat](https://docs.microsoft.com/en-us/azure/cognitive-services/qnamaker/how-to/chit-chat-knowledge-base?WT.mc_id=AI-MVP-5003073) * [New built-in entity types for people names and geographic locations in LUIS](https://azure.microsoft.com/en-us/blog/build-conversational-experiences-with-microsoft-bot-framework/) * [Microsoft releases platform to build company-specific skills for Cortana](https://blogs.microsoft.com/ai/cortana-for-enterprise/?utm_source=em&utm_campaign=78690) * [Announcing Speech service: A new service combining all speech capabilities in a single endpoint](https://azure.microsoft.com/en-us/blog/global-scale-ai-with-azure-cognitive-services/) * [Video Indexer hit GA status](https://azure.microsoft.com/en-us/blog/microsoft-azure-media-services-and-our-partners-welkom-you-to-ibc-2018/) --- title: "The Microsoft Cognitive Services Resource Cheat Sheet" url: https://bisser.io/the-microsoft-cognitive-services-resource-cheat-sheet/ markdown: https://bisser.io/the-microsoft-cognitive-services-resource-cheat-sheet/index.md date: 2018-09-01 lastmod: 2018-09-01 tags: AI, Bot Framework categories: AI, Cognitive Services description: "The Microsoft Cognitive Services Resource Cheat Sheet" status: legacy reading_time_min: 1 --- In the "jungle" of documentation and resources on the internet nowadays, it's tough to keep the overview and know where to look for in order to seek new information about certain topics. Microsoft is now trying to unify the documentation across their products in order to have 1 place to go for. But until that is fully established, we need to conduct multiple resources. To overcome this issue in the field of the Azure Bot Service, I already released a resource cheat sheet in order to have all important sources in your pocket. Now I did the same thing for Microsoft's Cognitive Services, in order to help you find the information you need quicker, get yours here: {{< figure src="/images/Cloudguy_Cognitive_Services_Resource_Cheat_Sheet-1-1024x619.png" >}} You can also use the link list provided below: * **[Cognitive Services Docs](https://docs.microsoft.com/azure/cognitive-services?WT.mc_id=AI-MVP-5003073)** * **[Cognitive Services](http://aka.ms/cognitive)** * **[AI School](https://aischool.microsoft.com)** * **[GitHub Intelligent Kiosk Sample](http://aka.ms/intelligent-kiosk)** * **[Feedback & Support](https://cognitive.uservoice.com)** The full image can be downloaded here: [Microsoft Cognitive Services Resource Cheat Sheet](https://cloudguy.blob.core.windows.net/resources/infographics/Cloudguy_Cognitive_Services_Resource_Cheat_Sheet.png) Feel free to use it for internal use in your presentations or documents. The only point I ask you to do is to credit me and link this article to your documents where you use it. Please do not sell this as your own 😉 --- title: "The Microsoft Bot Framework Resource Cheat Sheet" url: https://bisser.io/the-microsoft-bot-framework-resource-cheat-sheet/ markdown: https://bisser.io/the-microsoft-bot-framework-resource-cheat-sheet/index.md date: 2018-07-30 lastmod: 2018-07-30 tags: AI, Bot Framework categories: AI, Bot Framework description: "The Microsoft Bot Framework Resource Cheat Sheet" status: legacy reading_time_min: 1 --- Nowadays, there is a ton of information out there for every service Microsoft has released, which is a good thing. But there are times when you search for something special, but you can't find it as there are too many sites and resources which you need to crawl to find what you are looking for. The same is true for the Azure Bot Service and the MS Bot Framework, as there is plenty of documentation, but it is spread over a couple of different sites. Therefore, the following cheat sheet should help you find the information you are looking for quicker to spend more time on the things you need to fulfill... {{< figure src="/images/Cloudguy_Bot_Framework_Resource_Cheat_Sheet.png" >}} You can also use the link list provided below: * **[Azure Bot Service Docs](https://aka.ms/azure/bots/overview/docs)** * **[Microsoft Bot Framework](https://aka.ms/botframework)** * **[AI School](https://aischool.microsoft.com)** * **[GitHub BotBuilder SDK](https://github.com/Microsoft/botbuilder)** * **[Feedback & Support](https://aka.ms/azure/bots/overview/feedback)** The full image can be downloaded here: **[MS Bot Framework Resource Cheat Sheet](https://cloudguy.blob.core.windows.net/resources/infographics/Cloudguy_Bot_Framework_Resource_Cheat_Sheet.png)** Feel free to use it for internal use in your presentations or documents. The only point I ask you to do is to credit me and link this article to your documents where you use it. Please do not sell this as your own 😉 --- title: "QnAMaker.ai Cheat Sheet" url: https://bisser.io/qnamaker.ai-cheat-sheet/ markdown: https://bisser.io/qnamaker.ai-cheat-sheet/index.md date: 2018-07-10 lastmod: 2018-07-10 tags: AI, Bot Framework categories: AI, Cognitive Services description: "QnAMaker.ai Cheat Sheet" status: legacy reading_time_min: 1 --- **_"Copy, paste... Bot!"_** - This is the slogan of the **[QnAMaker.ai](https://www.qnamaker.ai/)** service, which is one of the **[Cognitive Services](http://aka.ms/cognitive-services)** in the knowledge category. This service offers a very easy-to-use toolset for creating, populating and publishing knowledgebases for FAQs which can be consumed by the **[Azure Bot Service](https://azure.microsoft.com/en-us/services/bot-service/)** very easy to build smart QnA Bots in a very short period of time. As the creation of such a QnAMaker knowledgebase is straightforward, the following infographic should help illustrate the process of creating, populating, testing and publishing a KB through the QnAMaker service. > From FAQ to Bot in minutes. Build, train and publish a simple question and answer bot based on FAQ URLs, structured documents, product manuals or editorial content in minutes[...](https://www.qnamaker.ai/) {{< figure src="/images/Cloudguy_QnAMaker_cheatsheet-1024x589.png" >}} The full image can be downloaded here: **[QnAMaker.ai Cheatsheet](https://cloudguy.blob.core.windows.net/resources/infographics/Cloudguy_QnAMaker_cheatsheet.png)** Feel free to use it for internal use in your presentations or documents. The only point I ask you to do is to credit me and link this article to your documents where you use it. Please do not sell this as your own 😉 --- title: "Cognitive Services Infographic [UPDATED]" url: https://bisser.io/cognitive-services-infographic-updated/ markdown: https://bisser.io/cognitive-services-infographic-updated/index.md date: 2018-06-25 lastmod: 2018-06-25 tags: AI, Bot Framework categories: AI, Cognitive Services description: "Cognitive Services Infographic [UPDATED]" status: legacy reading_time_min: 1 --- A couple of months ago, I did [a post about the Microsoft Cognitive Services](https://www.cloudguy.pro/cognitive-services-101-infographic/) with an infographic attached in order to better understand the various services and offerings in that area. Now a couple of months later, some things changed in that field, especially the offerings and the services (some disappeared, some have been merged and some appeared). Therefore, I updated my infographic in order to reflect the current services which are available: {{< figure src="/images/Cloudguy_Cognitive-Services-Infographic_2018.png" >}} The full image can be downloaded [here...](https://cloudguy.blob.core.windows.net/resources/infographics/Cloudguy_Cognitive-Services-Infographic_2018.png?st=2018-06-25T08%3A29%3A00Z&se=2036-06-26T08%3A29%3A00Z&sp=rl&sv=2017-07-29&sr=b&sig=LGbyrzAN0FQ5WKwdxedMrW2r3FZ7Cl0TGFiejrE0Z0c%3D) Feel free to use it for internal use in your presentations or documents. The only point I ask you to do is to credit me and link this article in your documents where you use it. Please do not sell this as your own 😉 --- title: "New Bot Builder Tools [Infographic]" url: https://bisser.io/new-bot-builder-tools-infographic/ markdown: https://bisser.io/new-bot-builder-tools-infographic/index.md date: 2018-05-22 lastmod: 2018-05-22 tags: AI, Bot Framework categories: AI, Bot Framework description: "New Bot Builder Tools [Infographic]" status: legacy reading_time_min: 2 --- A couple of weeks ago at the Microsoft Build conference (see [this post](https://www.cloudguy.pro/azure-ai-platform-updates-build-2018/) for details on the announcements concerning AI), there were some cool new tools for the Bot Builder announced. These tools are already in preview and can be downloaded [here](https://github.com/Microsoft/botbuilder-tools). The following table should give an overview of the new tools released: | Tool | Description | | --- | --- | | [Chatdown](https://github.com/Microsoft/botbuilder-tools/tree/master/Chatdown) | Prototype mock conversations in markdown and convert the markdown to transcripts you can load and view in the new V4 Bot Framework Emulator | | [MSBot](https://github.com/Microsoft/botbuilder-tools/tree/master/MSBot) | Create and manage connected services in your bot configuration file | | [LUDown](https://github.com/Microsoft/botbuilder-tools/tree/master/Ludown) | Build LUIS language understanding models using markdown filese | | [LUIS](https://github.com/Microsoft/botbuilder-tools/tree/master/LUIS) | Build LUIS language understanding models using markdown files | | [QnAMaker](https://github.com/Microsoft/botbuilder-tools/tree/master/QnAMaker) | Create and manage [QnAMaker.ai](http://qnamaker.ai/) Knowledge Bases | | [Dispatch](https://github.com/Microsoft/botbuilder-tools/tree/master/Dispatch) | Build language models allowing you to dispatch between disparate components (such as QnA, LUIS and custom code) | | [LUISGen](https://github.com/Microsoft/botbuilder-tools/tree/master/LUISGen) | Autogenerate backing C#/Typescript classes for your LUIS intents and entities | In order to visualize the tools and use cases you can use the following infographic which should explain these new tools and features a bit better by visualizing them: {{< figure src="/images/Cloudguy_Bot_Builder_tools-1024x623.webp" >}} Feel free to use it for internal use in your presentations or documents. The only point I ask you to do is to credit me and link this article to your documents where you use it. Please do not sell this as your own 😉 --- title: "Azure AI Platform - Updates from Build 2018" url: https://bisser.io/azure-ai-platform-updates-from-build-2018/ markdown: https://bisser.io/azure-ai-platform-updates-from-build-2018/index.md date: 2018-05-10 lastmod: 2018-05-10 tags: AI, Bot Framework categories: AI, Bot Framework description: "Azure AI Platform - Updates from Build 2018" status: legacy reading_time_min: 2 --- Microsoft's annual BUILD conference took again place from 7th - 9th May 2018 with a lot of new and awesome updates for the Azure AI platform. The following list should give you a short overview of all news and announcements: * **Bot Builder SDK v4 now in preview:** * [Bot Builder SDK v4 .NET](https://github.com/Microsoft/botbuilder-dotnet) * [Bot Builder SDK v4 JavaScript](https://github.com/Microsoft/botbuilder-js) * [Bot Builder SDK v4 Python](https://github.com/Microsoft/botbuilder-python) * [Bot Builder SDK v4 Java](https://github.com/Microsoft/botbuilder-java) * **Bot Framework Emulator v4 now in preview:** * Updates and Improvements * "The new emulator includes features like an improved dynamic layout model, support for multiple bot configurations, simplified bot components management, and the ability to inspect responses from and deep link to connected services such as LUIS and QnA." [Read more...](https://blog.botframework.com/2018/05/07/microsoft-conversational-ai-tools-enable-developers-to-build-connect-and-manage-intelligent-bots/) * [Download here](https://github.com/Microsoft/BotFramework-Emulator/releases) * **New Bot Builder tools now in preview:** * [Chatdown](https://github.com/Microsoft/botbuilder-tools/tree/master/Chatdown) - "Prototype mock conversations in markdown and convert the markdown to transcripts you can load and view in the new V4 Bot Framework Emulator" * [MsBot](https://github.com/Microsoft/botbuilder-tools/tree/master/MSBot) - "Create and manage connected services in your bot configuration file" * [LUDown](https://github.com/Microsoft/botbuilder-tools/tree/master/Ludown) - "Build LUIS language understanding models using markdown files" * [LUIS](https://github.com/Microsoft/botbuilder-tools/tree/master/LUIS) - "Create and manage your [LUIS.ai](http://luis.ai/) applications" * [QnAMaker](https://github.com/Microsoft/botbuilder-tools/tree/master/QnAMaker) - "Create and manage [QnAMaker.ai](http://qnamaker.ai/) Knowledge Bases" * [Dispatch](https://github.com/Microsoft/botbuilder-tools/tree/master/Dispatch) - "Build language models allowing you to dispatch between disparate components (such as QnA, LUIS and custom code)" * [LUISGen](https://github.com/Microsoft/botbuilder-tools/tree/master/LUISGen) - "Autogenerate backing C#/Typescript classesfor your LUIS intents and entities" * **LUIS updates:** * Regex entities - "Regex entities allow the identification of an entity in the utterance based on a regular expression" [Read more...](https://azure.microsoft.com/en-us/blog/full-integrated-experience-simplifying-language-understanding-in-conversational-ai-systems/) * Patterns - "Patterns enable developers to effectively define intents without having to provide many utterances. Patterns could also encompass entities with variable length represented as Patterns.any entities" [Read more...](https://azure.microsoft.com/en-us/blog/full-integrated-experience-simplifying-language-understanding-in-conversational-ai-systems/) * **QnAMaker.ai is now GA** * Now the [QnAMaker.ai](https://www.qnamaker.ai/) has a fixed pricing tier structure and full support options - go ahead and [get started](https://docs.microsoft.com/en-us/azure/cognitive-services/qnamaker/overview/overview?WT.mc_id=AI-MVP-5003073) * **Cognitive Services updates:** * There have been some updates announced for various Cognitive Service APIs - please check out [this link](https://azure.microsoft.com/en-us/blog/microsoft-empowers-developers-with-new-and-updated-cognitive-services/) for details * **Project Conversation Learner:** * This is one of the new labs in the Cognitive Services portfolio, which lets you build conversational interfaces like bots that learn from example interactions. [Read more...](https://labs.cognitive.microsoft.com/en-us/project-conversation-learner) * **Project Personality Chat:** * This lab enhances your bot by adding small talk capabilities out-of-the box which makes it easier for you to build a comprehensive bot which is capable of doing smalltalk either professional, friendly or even humorous. [Read more...](https://labs.cognitive.microsoft.com/en-us/project-personality-chat) --- title: "Build your very own Twitter QnA Bot" url: https://bisser.io/build-your-very-own-twitter-qna-bot/ markdown: https://bisser.io/build-your-very-own-twitter-qna-bot/index.md date: 2018-03-15 lastmod: 2018-03-15 tags: AI, Bot Framework categories: AI, Bot Framework description: "Build your very own Twitter QnA Bot" status: legacy reading_time_min: 4 --- The idea of this solution has come to my mind a couple of months ago when doing a couple of conference sessions. During the sessions, I had some questions coming from the audience, which is a good thing for me, but somehow I noticed, that those a more or less general questions rather than deep technical ones. Therefore, I thought it would be helpful for me as well as for the audience to have a much simpler way of asking me questions during speaking engagements, but also whenever someone has a question which might be answered by me. And which social media platform would be suited better than Twitter for exchanging short messages which might be of interest to a broader audience? So the idea of my #AskCloudguy Twitter Bot was born. Feel free to implement your very own Twitter Bot based on a specific hashtag in order to build an automated Twitter QnA service for your followers based on the following solution. ## Solution Overview The main architecture of this solution is basically reflected by the following illustration: {{< figure src="/images/Cloudguy_twitterbot_solution_design-1024x571.png" >}} The main idea is that there is an Azure Logic App which is checking for new tweets based on a given hashtag. Whenever the Azure Logic App gets triggered, it receives the tweet text as well as the tweet ID, by the Twitter connector. The tweet text is forwarded to the [QnAMaker.ai](https://qnamaker.ai) service in order to generate an answer based on the tweet text. After the answer is received by the QnAMaker the Azure Logic App makes an HTTP POST request to the Azure Function and forwards the tweet ID as well as the answer in the HTTP body in order to reference the original tweet and being able to post a reply to the tweet rather than posting a new tweet. The Azure Function itself connects to the Twitter API via the Twit module and updates the original tweet by referencing the tweet ID and replies with the answer generated by the QnAMaker service. ## QnA Maker Service The only thing you have to do is to create a new knowledgebase within the [QnAMaker](https://qnamaker.ai) service and populate it with QnA pairs: {{< figure src="/images/Cloudguy_twitterbot_qnamaker-1024x487.png" >}} ## Azure Function The Azure Function is the service where the actual Twitter reply action happens. In order to connect to the Twitter API I decided to use the Node module [Twit](https://github.com/ttezel/twit) which needs to be installed by running the following command in the function's console first: ```shell npm install twit ``` After the module and it's dependencies have been installed, it's time to update the application settings of our Azure Function, but first we need to gather the keys and secrets by Twitter. So head over to the [Twitter app portal](https://apps.twitter.com/) and create a new app in order to gather the following keys and secrets: * consumer_key * consumer_secret * access_token * access_token_secret After you have copied them somewhere, we need to add those four variables to our application settings of the Azure Function in order to keep these secrets and keys out of our code: {{< figure src="/images/Cloudguy_twitterbot_azure_function_app_settings-1024x518.png" >}} Now that we have that in place, we can update/replace the code of the function by the following code: ```JavaScript module.exports = function (context, req) { context.log('JavaScript HTTP trigger function processed a request.'); var Twit = require('twit'); var T = new Twit({ consumer_key: process.env['consumer_key'], consumer_secret: process.env['consumer_secret'], access_token: process.env['access_token'], access_token_secret: process.env['access_token_secret'], timeout_ms: 60*1000, // optional HTTP request timeout to apply to all requests. }); if (req.query.name || (req.body && req.body.tweetID)) { context.res = { status: 200, /* Defaults to 200 */ body: "Status " + (req.query.name || req.body.tweetID) } // create the reply object var new_tweet = req.body.answer; var tweet = { status: new_tweet, in_reply_to_status_id: req.body.tweetID } // reply to the original tweet T.post('statuses/update', tweet, tweeted); function tweeted(err, data, response) { if (err) { context.log(err); } else { context.log("It worked!"); } } } else { context.res = { status: 400, body: "Please pass a name on the query string or in the request body" }; } context.done(); }; ``` ## Azure Logic App Design The design of the Azure Logic App looks as follows: {{< figure src="/images/Cloudguy_twitterbot_azure_logic_app_design.png" >}} Feel free to implement your very own hashtag. Please make sure to replace the Uri of the Azure Function with the correct Uri of your Azure Function. Now we have everything in place we can test the solution by posting a new tweet with a question which is part of a QnA pair in our QnA knowledge base and check if the reply is being posted to that tweet accordingly: {{< figure src="/images/Cloudguy_twitterbot_tweet-1024x667.png" >}} Now that's it. You build your very own QnA Bot for your very own Twitter hashtag. If you have any questions please feel free to contact me - try it out by posting a new question on Twitter with the hashtag #AskCloudguy and my Bot will reach out to you ;) --- title: "Migrate your Bot Framework or Azure Bot Service preview bot to Azure Bot Service the right way" url: https://bisser.io/migrate-your-bot-framework-or-azure-bot-service-preview-bot-to-azure-bot-service-the-right-way/ markdown: https://bisser.io/migrate-your-bot-framework-or-azure-bot-service-preview-bot-to-azure-bot-service-the-right-way/index.md date: 2018-03-08 lastmod: 2018-03-08 tags: AI, Bot Framework categories: AI, Bot Framework description: "Migrate your Bot Framework or Azure Bot Service preview bot to Azure Bot Service the right way" status: legacy reading_time_min: 3 --- Microsoft has GAed the **Azure Bot Service** a couple of weeks ago, which was a huge step forward for all of us who are doing bots and **AI** apps in the Microsoft ecosystem. With this step, Microsoft is also pushing one more area into the Azure portal which is bots. Remember, during the preview, you had to create your bot in the [Bot Framework portal](https://dev.botframework.com) and add your channels and everything related to your bot in that specific portal. Now with the transition to the GA status, they migrated that management tasks into the Azure portal, which definitively makes sense as the bots run in **Azure**. With that move, Microsoft also informed the customers that they need to manually "migrate" the bots, which have been created in the Bot Framework portal into Azure Bot Service bots which will be fully managed in the Azure portal. The migration flow as it is can be seen in the following steps: ## 1\. Bot Framework Portal - Start Migration When you log in to the Bot Framework portal, you simply need to choose the bot which you want to migrate and click migrate: {{< figure src="/images/capture20180306143116967.png" >}} After clicking migrate, it will display you some basic information about your Azure environment: {{< figure src="/images/capture20180306143231446.png" >}} Now you need to accept the new terms of service (as usual) ;) {{< figure src="/images/capture20180306143250799-1024x252.png" >}} And once your bot service migration is finished (which can take a couple of minutes), you will be presented with the following screen: {{< figure src="/images/capture20180306143557389.png" >}} ## 2\. Azure Portal - New Bot Management tools Now your bot is fully migrated to the Azure Bot Service and it should be in the list of bots in the Azure portal, where you can click on it to manage it. When you view your bot's details in the portal you will see the following information: {{< figure src="/images/capture20180306143706148-1024x193.png" >}} > This bot is migrated from the old Azure Bot Service. You can rollback the migration in order to have access to your old bot before 03/31/2018\. After 03/31/2018, the old service will be deprecated. This can be done by browsing to the Bot Framework portal and hitting "**Roll back migration**": {{< figure src="/images/capture20180306143722685-1024x274.png" >}} This means that you can roll back your bot's migration until the end of March, in case you have any problems with your bot after the migration. And I'm pretty sure you will run into errors when you have a bot which is running NodeJS. As I tried to test my bot in the web chat window from the Azure Portal, I faced the following issue after the migration "**couldn't send retry**": {{< figure src="/images/capture20180308080740780.png" >}} So I debugged the bot and quickly found the error in the debug console: {{< figure src="/images/capture20180308080726156-1024x205.png" >}} As the bot has been migrated, you also need to re-install all your dependencies (at least for NodeJS bots) in order to have them act properly again. So the easiest way to do this is to navigate to the Kudu console (which can be found on your bot's advanced settings page in the Azure Portal "Advanced Tools") or you can simply navigate to [https://.scm.azurewebsites.net/](https://.scm.azurewebsites.net/debugconsole) debugconsole and there you have to navigate "**D:\home\site\wwwroot\messages**" and execute "npm**install**" as follows: {{< figure src="/images/capture20180308081324355-1024x392.png" >}} After the installation of the dependencies have been successfully installed you should be good to go to test your bot again and it should be up and running: {{< figure src="/images/capture20180308083552589-1024x221.png" >}} This is something quite obvious but you might forget about it when migrating and therefore it could cause a lot of problems, when your bot is not responding... --- title: "Solving the problem of too much lines of code for a fresh Azure Functions Bot" url: https://bisser.io/solving-the-problem-of-too-much-lines-of-code-for-a-fresh-azure-functions-bot/ markdown: https://bisser.io/solving-the-problem-of-too-much-lines-of-code-for-a-fresh-azure-functions-bot/index.md date: 2018-02-08 lastmod: 2018-02-08 tags: AI, Bot Framework categories: AI, Bot Framework description: "Solving the problem of too much lines of code for a fresh Azure Functions Bot" status: legacy reading_time_min: 2 --- Have you ever faced the issue of looking at more than 200.000 lines of code after creating a fresh**Functions Bot** via the**Azure Portal**? Well, I have unfortunately and I didn't know where to put my code in this massive amount of **NodeJS** code. As a lightweight developer, I am not that familiar in reading someone else's code so I didn't want to spend hours in studying the code below in order to get an idea where to put my code into. But I found a solution to this problem which you'll see below. {{< figure src="/images/capture20180206144142729.png" >}} But then I started debugging. Well, not the code but the Azure Portal ;) And after a while, I found the solution to my problem. Looking at the navigation pane to the right in the portal you can view the solution's files. And when you create a bot via the Azure Portal with a **NodeJS** template it usually comes with the following files: {{< figure src="/images/capture20180206144209891-1024x333.png" >}} Now if you click on the **index.js** file which is apparently already open it gets reloaded in the portal and somehow the 200.000 lines of meaningless (at least to me meaningless) code transform into 42 lines of meaningful code, isn't that awesome? ;) {{< figure src="/images/capture20180206144228825-1024x673.png" >}} So now you should be able to start adding more lines of code in there to actually develop your bot's logic… Happy coding! If you still face this issue feel free to get in touch with me... --- title: "Boost your Modern Workplace with Microsoft's AI ecosystem - Part 1 - SharePoint Online Intranet Bot" url: https://bisser.io/boost-your-modern-workplace-with-microsofts-ai-ecosystem-part-1-sharepoint-online-intranet-bot/ markdown: https://bisser.io/boost-your-modern-workplace-with-microsofts-ai-ecosystem-part-1-sharepoint-online-intranet-bot/index.md date: 2018-02-06 lastmod: 2018-02-06 tags: AI, Bot Framework categories: AI, Bot Framework description: "Boost your Modern Workplace with Microsoft's AI ecosystem - Part 1 - SharePoint Online Intranet Bot" status: legacy reading_time_min: 8 --- The terms **"Modern Workplace"** or **"Digital Workplace"** is something every one of us have heard about in the last couple of months probably. It is some trend which should improve the workplace situation in order to digitally transform the way we work and make it more easier to communicate and collaborate with others. Therefore, the use of smart services can be beneficial when it comes to really transforming your workplace into a modern one. As Microsoft offers a variety of services & solutions in the AI world, these services can fit perfectly fine into that new world which helps your users adopt the new workplace easier. Therefore, this blog post is the first in series with much more to come which is dealing with the question how to actually integrate those AI services in the **Microsoft AI ecosystem** into your modern workplace to make it smarter and help your users in terms of adoption. In this first article, we will have a look at one major topic when talking about the modern workplace which is the "Intranet" and how to integrate some kind of digital assistant in the form of a bot into your SharePoint (Online) based intranet. # Prerequisites * In order to implement the scenario into a SharePoint Online environment we need the following things: * Office 365 Tenant (with global admin or at least SharePoint Online service admin permissions) * Azure Subscription * Development environment set up according to [Set up your SharePoint Framework development environment](https://docs.microsoft.com/en-us/sharepoint/dev/spfx/set-up-your-development-environment) # Bot Design As the Bot is our key component in this scenario we want to start building it. Therefore we have to go to the Azure Portal click "+ New" - Search for "Bot" and select "Functions Bot" as this is the serverless approach of building and deploying bots. {{< figure src="/images/capture20180206073323670.png" >}} After clicking "Create" we have to fill out some basic properties in order to give the bot a name, select the right subscription, region and template. My bot has the following specs, but please feel free to choose whatever you want (Attention: Stick with the "Consumption Plan" for the Hosting Plan as this defines that you actually pay what you use - so if you don't use anything as you might have turned off your bot you won't be charged. Please read [Azure Functions scale and hosting](https://docs.microsoft.com/en-us/azure/azure-functions/functions-scale) for more details on the hosting plans). {{< figure src="/images/capture20180206073323670.png" >}} ![Bot creation in the Azure Portal](/static/img/capture20180206073925125-1024x555.png) As you can see I have gone for the NodeJS Question and Answer template as I feel comfortable in building NodeJS apps. But you can, of course, go for the C# template as well if you like. The reason why I chose the QnA template is simple - it comes with a predefined code base which has everything defined for connecting to the QnA Maker API (qnamaker.ai) which can be used to easily build a set of question and answer pairs as this is most probably a good starting point for an intranet bot. After your bot has been created we can navigate to the resource in the Azure Portal in order to start building it. But first of all, we need to define our QnA knowledge base. So we need to navigate to [QnAMaker](https://qnamaker.ai) where we want to create a new service and fill out a basic form: {{< figure src="/images/capture20180206073323670.png" >}} ![QnAMaker service creation](/static/img/capture20180206080521618.png) As you can see I have left the URL(S) blank as I don't have a website hosting some kind of FAQ pages at the moment. You can, of course, add your URL(S) in here if you have some sort of knowledge base already or you can select "STARTING FROM SCRATCH" where you have the chance to enter your question and answer pairs manually on the next page. But you can also use my sample file which you can download [here](https://1drv.ms/u/s!AjkHfQk6ArWTh-JzQqbBMVgGGT5R7Q) in order to have at least some basic questions and answers which might be asked by your users (please feel free to adapt it according to your needs). After filling out the form and clicking create you will be presented with your QnA pairs. You now have the chance to modify them or to add new ones, but basically, we are ready to train & publish our service. So the first thing is to click on "Save and retrain" if you made any modifications (if you didn't you can skip this) followed by publishing it. After you have published it you will be presented with a sample HTTP request which can be used in order to build your bot. But what we actually need now are the 2 values of the sample: * QnAKnowledgebaseId * QnASubscriptionKey The QnAKnowledgebaseId can be found in the first line after POST /knowledgebases/ and the QnASubscriptionKey is the value provided after Ocp-Apim-Subscription-Key. Please copy both values somewhere as we need them in the next step. Now we need to go back to the Bot's page in the Azure Portal and navigate to the "Application Settings" pane where we need to insert the values we copied before: {{< figure src="/images/capture20180206081337300-1024x555.png" >}} After inserting the values and saving we are now ready to build our bot. But wait - we do not need to build our bot now because the bot is already ready for use. Go ahead and test the bot directly in the Azure Portal: {{< figure src="/images/capture20180206081614583-1024x555.png" >}} The cool thing is that the QnA Maker is so smart that it even renders email addresses correctly to make them actually usable without any modification. So now we have our bot ready for deployment into our SharePoint Online based intranet. As we will use Direct Line for the communication between our SharePoint Online page and the bot we need to enable the Direct Line channel on the Bot's channel settings page and make sure to copy the secret key and store it somewhere as we will need that later on: {{< figure src="/images/capture20180206082347467-1024x555.png" >}} # SharePoint Online Setup I have set up a SPO communication site as this might reflect the basic ideas of a modern intranet build with SharePoint Online. So now we have to get our Bot deployed to our SPO communication site. Therefore we will make use of one of the SPFX sample web parts build by [Joel Rodrigues](https://github.com/joelfmrodrigues) and [Vesa Juvonen](https://github.com/VesaJuvonen) (Thanks for that - Great work!!) as this can be used right away. So go ahead and download the sample from [GitHub](https://github.com/SharePoint/sp-dev-fx-webparts/tree/master/samples/react-bot-framework) onto your development machine and follow the minimal path to awesome on this page to successfully use it. So after running gulp serve in your command line you should be presented with the local SharePoint workbench where you can start adding the web part to the page (Please make sure you have added your Direct Line Secret in the web part's config to make it actually communicate with the Bot): {{< figure src="/images/capture20180206083321847-1024x555.png" >}} Now we are actually ready to deploy the web part to our SPO environment. For the sake of simplicity I won't show you how to make it enterprise ready and deploy our web part's packages and dependencies to a CDN to fully host it in the cloud, but if you are interested in doing so, then please take a look at [Host your client-side web part from Office 365 CDN](https://docs.microsoft.com/en-us/sharepoint/dev/spfx/web-parts/get-started/hosting-webpart-from-office-365-cdn) and follow these steps mentioned there. Going back to the command line window, we can now safely stop the gulp serve task (CTRL+C) and execute the following command gulp package-solution in order to package the solution. This will create a folder called **"**sharepoint**"** in your solution as well as a .sppkg file. Now go ahead to your SPO app catalog and upload your package there (as described here: [Deploy your client-side web part to a SharePoint page](https://docs.microsoft.com/en-us/sharepoint/dev/spfx/web-parts/get-started/serve-your-web-part-in-a-sharepoint-page)): {{< figure src="/images/upload-solution-app-catalog-1024x388.png" >}} Now go back to your communication site and select add an app from the gear in the right upper corner. Now search for the web part we uploaded and click on it in order to add it to the page. Upon completion, we can now add the web part to our site finally. Now edit the page add your web part to the page where you want the bot to be placed, fill in the Direct Line Secret in the web part's config section and make sure you run the gulp serve task from your command line. If you have done everything you should be able to chat with the bot right inside you SharePoint Online communication site. {{< figure src="/images/capture20180206090342667-1024x514.png" >}} Congratulations you now have your very own digital assistant deployed to your intranet without much development effort (Of course you could now adapt the styling and brand the web part to fit your CI/CD - but that's something you may be better in than me). # Conclusion * The [Azure Bot Service](https://azure.microsoft.com/en-us/services/bot-service/) offers great ways for building, hosting and deploying bots without the need to have a development background * [QnAMaker](https://qnamaker.ai) is one great way of adding some intelligence to a digital assistant in form of a bot * Bots make intranets look and behave more modern and will help your users to adopt the new technologies of your modern workplace portfolio quicker ## Kudos Thanks to [Vesa Juvonen](https://twitter.com/vesajuvonen), the whole [Office Dev PnP](https://twitter.com/officedevpnp) team and the SPFX team for pushing so many great features. Another shout out goes to [Thomas Gölles](http://www.modernworkplacesolutions.rocks) for helping me out with the development skills (I don't have) ;) as well as the whole [Bot Framework](https://twitter.com/msbotframework) team for enabling us non-devs building great bots without much development effort... --- title: "LUIS.ai Cheatsheet" url: https://bisser.io/luis.ai-cheatsheet/ markdown: https://bisser.io/luis.ai-cheatsheet/index.md date: 2018-01-24 lastmod: 2018-01-24 tags: AI, Cognitive Services categories: AI, Cognitive Services description: "LUIS.ai Cheatsheet" status: legacy reading_time_min: 1 --- With the announcement of the GA status of the **[Language Understanding Intelligent Service](https://azure.microsoft.com/en-us/services/cognitive-services/language-understanding-intelligent-service/)** a couple of weeks ago, I was thinking about how to easily enable people in using this awesome Cognitive Service. I was asked many times on how to easily use that service but I faced a lot of open questions and confusion. That's why I decided to create a cheat sheet to illustrate the process of creating, training and publishing a LUIS model in order to use it within a Bot or other apps which should act intelligent and understand the human language. > A machine learning-based service to build natural language understanding into apps, bots, and IoT devices. Quickly create enterprise-ready, custom models that continuously improve [[...]](https://azure.microsoft.com/en-us/services/cognitive-services/language-understanding-intelligent-service/) {{< figure src="/images/cloudguy_LUIS_cheatsheet.png" >}} The full image can be downloaded here: [**LUIS.ai cheat sheet**](https://cloudguy.blob.core.windows.net/resources/infographics/Cloudguy_LUIS_cheatsheet.png) Feel free to use it for internal use in your presentations or documents. The only point I ask you to do is to credit me and link this article to your documents where you use it. Please do not sell this as your own 😉 --- title: "Microsoft AI Platform Infographic" url: https://bisser.io/microsoft-ai-platform-infographic/ markdown: https://bisser.io/microsoft-ai-platform-infographic/index.md date: 2017-12-26 lastmod: 2017-12-26 tags: AI, Bot Framework categories: AI, Bot Framework description: "Microsoft AI Platform Infographic" status: legacy reading_time_min: 2 --- _"Artificial intelligence productivity for every developer and every scenario"_ - this is the slogan which is shown on the landing page of the **[Microsoft AI Platform](https://azure.microsoft.com/en-us/overview/ai-platform/)**. But many people don't know which services are included in this broad AI ecosystem. It is not only about **Bots** or **Machine Learning** - it's about much more like **Cognitive Services**, it's about the tools used for developing AI apps and the frameworks underneath all those great and intelligent services. That's why the following infographic should show all the services which are included in the Microsoft AI Platform to show the various solutions which can be used to develop intelligent applications. {{< figure src="/images/AI_platform_Infographic-1024x616.png" >}} > With the Azure platform and productivity services, you can create the next generation of applications that span an intelligent cloud as well as an intelligent edge powered by AI. Use a comprehensive set of flexible [AI services](https://azure.microsoft.com/en-us/overview/ai-platform/#services) for any scenario, and enterprise-grade [AI infrastructure](https://azure.microsoft.com/en-us/overview/ai-platform/#infrastructure) that runs AI workloads anywhere at scale. Modern [AI tools](https://azure.microsoft.com/en-us/overview/ai-platform/#tools) designed for developers and data scientists help you create AI solutions easily, and with maximum productivity. [Read more...](https://azure.microsoft.com/en-us/overview/ai-platform/) The full image can be downloaded here: **[Microsoft AI Platform Infographic](https://github.com/cloudguy-pro/Infographics/blob/master/AI_platform_Infographic.png)** Feel free to use it for internal use in your presentations or documents. The only point I ask you to do is to credit me and link this article to your documents where you use it. Please do not sell this as your own 😉 --- title: "Azure Bot Service Couldn't send error on emulator" url: https://bisser.io/azure-bot-service-couldnt-send-error-on-emulator/ markdown: https://bisser.io/azure-bot-service-couldnt-send-error-on-emulator/index.md date: 2017-12-06 lastmod: 2017-12-06 tags: AI, Bot Framework categories: AI, Bot Framework description: "Azure Bot Service Couldn't send error on emulator" status: legacy reading_time_min: 2 --- Yesterday I wanted to deploy a new bot with the Azure Bot Service to start developing a new digital assistant within the Azure Bot Framework platform. But as I created my bot from the Azure portal I ran into a strange error upon testing the bot immediately, which you can see below: {{< figure src="/images/capture20171205145953029.png" >}} As you can see the bot's function started, but it didn't finish successfully and in the test window I got the error message **"couldn't send retry"** which was quite strange as I didn't modify the code at all. I started to check if it's something in my Azure's region and the subscription but I also failed when deploying a new bot in another region with another subscription. So I asked the Azure support team for help and they immediately helped me solving the problem. Solution If you run into this issue, go to your bot's **package.**json file and check the botbuilder package version: {{< figure src="/images/capture20171206132616817.png" >}} If you see the same botbuilder version then please change it to **3.12.0** like shown below: {{< figure src="/images/capture20171206132654981.png" >}} Now you need to update your npm packages for your solution. So please go to [https://.scm.azurewebsites.net](https://your-bot-name.scm.azurewebsites.net) and click on **Debug console** and choose **CMD**. Now you have to navigate to **D:\home\site\wwwroot\messages** where you need to execute npm**update** like shown below: {{< figure src="/images/capture20171206132654981.png" >}} When this operation has finished, you should be fine now and your bot should be starting to work now and you can develop and test it from now on, have fun! Kudos to the **[Azure Support Team](https://twitter.com/AzureSupport)** and **[Andrea Orimoto](https://stackoverflow.com/users/6917329/andrea-orimoto) **who immediately helped me with my problem! If you still face the problem mentioned please refer to [this](https://stackoverflow.com/questions/47661295/azure-bot-service-couldnt-send-retry-on-emulator?noredirect=1#comment82292876_47661295) --- title: "Bot Framework – 101 (with Infographic)" url: https://bisser.io/bot-framework-101-with-infographic/ markdown: https://bisser.io/bot-framework-101-with-infographic/index.md date: 2017-11-20 lastmod: 2017-11-20 tags: AI, Bot Framework categories: AI, Bot Framework description: "Bot Framework – 101 (with Infographic)" status: legacy reading_time_min: 3 --- **[Microsoft's Bot Framework](https://dev.botframework.com)** is a great platform for designing, developing and deploying bots for various use cases. As nowadays many people are familiar with bots and they get used to connect and communicate with bots via social media, it's a good opportunity to bring a service to the next level by adding a bot to it for a better communication service. In the past, t was very hard to achieve that as you would basically have to develop the bot on your own and connect all the different systems and platforms together to make the bot available to a broader audience. With the Bot Framework Microsoft released a platform where you only need to handle your bot's design and logic without thinking about the various systems and services in which your bot should be communicating with your users. > Think of a bot as an app that users interact with in a conversational way. Bots can communicate conversationally with text, cards, or speech. A bot may be as simple as basic pattern matching with a response, or it may be a sophisticated weaving of artificial intelligence techniques with complex conversational state tracking and integration to existing business services. > The Bot Framework enables you to build bots that support different types of interactions with users. You can design conversations in your bot to be freeform. Your bot can also have more guided interactions where it provides the user choices or actions. The conversation can use simple text strings or more complex rich cards that contain text, images, and action buttons. And you can add natural language interactions, which let your users interact with your bots in a natural and expressive way. [Read more...](https://docs.microsoft.com/en-us/bot-framework/overview-introduction-bot-framework?WT.mc_id=AI-MVP-5003073) The following infographic should provide you with a high-level overview of the Bot Framework including all necessary components: * **Channels** * **Platforms & SDKs** * **Additional Services (Cognitive Services)** {{< figure src="/images/botframework_infographic-1024x403.jpg" >}} ## Channels **Channels** provide you with the ability to deploy your bot in **various systems** at the same time without the need to adapt your**business logic** within your bot. This makes it easy to build a bot which is capable of communicating with people via various channels and platforms to reach even more people with your bot. ## Platforms & SDKs Microsoft lets you build your bot with **C# or NodeJS** and host it on **Azure** as or on your own infrastructure. Therefore you are flexible and have the choice how to develop and host your bot without being bound to one specific way of implementation. ## Additional Services As your bot needs to be intelligent as well, you need to add services to your bot which make your bot smart. Therefore, the [**Cognitive Services**](https://bisser.io/cognitive-services-101-infographic/) provided by Azure let you add either **language understanding** or **voice recognition** services along much more to your bot to make the interaction with your users more human. * * * The full image can be downloaded [here](https://github.com/cloudguy-pro/Infographics/blob/master/MSBotFramework.png) Feel free to use it for internal use in your presentations or documents. The only point I ask you to do is to credit me and link this article to your documents where you use it. Please do not sell this as your own 😉 --- title: "Cognitive Services Text Analytics API GA" url: https://bisser.io/cognitive-services-text-analytics-api-ga/ markdown: https://bisser.io/cognitive-services-text-analytics-api-ga/index.md date: 2017-10-13 lastmod: 2017-10-13 tags: AI, Bot Framework categories: AI, Cognitive Services description: "Cognitive Services Text Analytics API GA" status: legacy reading_time_min: 1 --- One of the coolest **Cognitive Services APIs** is finally generally available: the **Text Analytics API**. If you have a look at **[Survey 365](http://www.cloudguy.pro/survey-365-an-intelligent-survey-solution/) **you will see what awesome stuff the Text Analytics API lets you do. Like in this example, you can extract text and detect the **sentiment score** of a given text to see if it is rather positive or negative. And all that without coding a lot. You can see the official announcement from the **Microsoft Azure** team about the **GA** information below: {{< figure src="/images/capture20171013073311325.png" >}} And by the way, if you are interested in a quick demo on how you can use the Text Analytics API just head over to** [Text Analytics API](https://azure.microsoft.com/en-us/services/cognitive-services/text-analytics/)** and enter your text to check the sentiment and extract the **key phrases** as I did below: {{< figure src="/images/capture20171013073731425.png" >}} --- title: "Cognitive Services and Bot Framework Visio Stencils" url: https://bisser.io/cognitive-services-and-bot-framework-visio-stencils/ markdown: https://bisser.io/cognitive-services-and-bot-framework-visio-stencils/index.md date: 2017-10-10 lastmod: 2017-10-10 tags: AI, Bot Framework categories: AI, Cognitive Services description: "Cognitive Services and Bot Framework Visio Stencils" status: legacy reading_time_min: 2 --- During the work on my first **[infographic on Cognitive Services](https://www.cloudguy.pro/cognitive-services-101-infographic/)** I had a hard time to get the right icons for all **Cognitive Services** and APIs as I used Visio to create it and there were no stencils for these services. So I decided to go with the icons, [Sandro Pereira](https://gallery.technet.microsoft.com/Collection-of-Integration-e6a3f4d0) created, which are totally awesome, but unfortunately, there are some service icons missing. So I had to create the missing icons on my own in Visio, which resulted in a huge amount of working hours to get this done. So I thought of creating a stencil set for all **AI** services and applications in the Microsoft AI ecosystem which I can use for all my future infographics. And as I'm probably not the only one who will need those icons to create awesome Visio graphics, I want to share those stencils with you all! So if you want to use them, please download the Visio stencils in my **[TechNet Gallery](https://gallery.technet.microsoft.com/Azure-AI-Visio-Stencils-55009cc9)** and create awesome graphics. To give you a quick glimpse of what you can expect from my stencils, the following image shows the current stencils and shapes for the various categories: ![Azure AI stencils (Cognitve Services & Bot Framework)](/static/img/Azure_AI_stencils.png) Azure AI stencils (Cognitive Services & Bot Framework) If you have any questions about the stencils or if you have improvement suggestions, please contact me via Twitter, email or any other social media channel as I would love to hear your feedback. Stay tuned for the next updated version of my stencil set which will include more awesome icons and shapes. --- title: "MSIgnite updates on AI at a glance" url: https://bisser.io/msignite-updates-on-ai-at-a-glance/ markdown: https://bisser.io/msignite-updates-on-ai-at-a-glance/index.md date: 2017-09-26 lastmod: 2017-09-26 tags: AI, Bot Framework categories: AI, Cognitive Services description: "MSIgnite updates on AI at a glance" status: legacy reading_time_min: 3 --- As the **[Microsoft Ignite](https://www.microsoft.com/en-us/ignite)** is currently going on in Orlando, there are a lot of news and updates on all the various areas. The main focus areas which have been presented in the keynote are: * Modern workplace * Business applications * Applications & infrastructure * **Data & AI** As you might already know I'm into all things **AI**, which is why I want to share the most recent and important updates on the AI ecosystem (**Cognitive Services**, **Bots** & **ML**) by Microsoft during the Ignite with you... * * * ## 25.09.2017 ### Service Updates * [**Text Analytics API**](https://azure.microsoft.com/en-us/services/cognitive-services/text-analytics/)**is now generally available**. Text Analytics is a cloud-based service that provides advanced natural language processing over raw text. It includes API functions such as sentiment analysis, key phrase extraction and language detection. * [**Bing Custom Search API**](https://azure.microsoft.com/en-us/services/cognitive-services/bing-custom-search/)**will be generally available in October**. Bing Custom Search lets you create a highly-customized targeted web search experience to deliver more relevant results from your targeted web space through a commercial grade service. * **[Bing Search APIs](https://azure.microsoft.com/en-us/services/cognitive-services/directory/#search) v7 will be will be generally available in October**. Allowing you to bring the immense knowledge of the planet to your applications, the v7 update will provide several improvements, such as results coming back fast with improved performance for queries on the **[Bing Web Search API](https://azure.microsoft.com/en-us/services/cognitive-services/bing-web-search-api/)**. New sorting and filtering options make it easier to find relevant results in news trending topics and image searches. Better error messages make it easy to troubleshoot and diagnose problem queries, and updated, modernized documentation make it easy bring the power of the Bing Search APIs to your applications. * Changes to the [**Language Understanding Intelligent Service**](https://azure.microsoft.com/en-us/services/cognitive-services/language-understanding-intelligent-service/) and [**Microsoft Bot Framework**](https://dev.botframework.com/)are planned, which contain everything you need to build and connect intelligent bots, **generally available later this year**. ### New Capabilities * **[QnAMaker](https://azure.microsoft.com/en-us/services/cognitive-services/qna-maker/)** preview API is now enabling to build, train and publish a simple question and answer bot from product manuals. * [**Face API**](https://azure.microsoft.com/en-us/services/cognitive-services/face/), [**Computer Vision API**](https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/) and [**Content Moderator**](https://azure.microsoft.com/en-us/services/cognitive-services/content-moderator/) are expanding in 7 additional regions - South Central US, West US2, East US, Brazil, North Europe, Australia East and East Asia. If you want to know more about these updates, take a look at [MSIgnite AI Updates 25.09.2017](https://azure.microsoft.com/en-us/blog/at-ignite-microsoft-is-updating-its-cognitive-services-collection-of-intelligent-apis/) But that's not enough. [Joseph Sirosh](https://twitter.com/josephsirosh) (Corporate Vice President for the Cloud AI group at Microsoft) also showed an important slide during his session **"The Microsoft AI platform"** which should give an overview of the AI platform within Azure: {{< figure src="/images/capture20170926081131373.png" >}} * * * ## 26.09.2017 ### Service Updates * Azure Machine Learning: Adding support for **Azure Batch AI** (Roadmap) * Azure Machine Learning: Rolling out to more regions (Roadmap) * Azure Machine Learning: Onward to **GA** (Roadmap) There have not been any announcements on service updates regarding**Cognitive**Services unfortunately. But what the **Bot Framework** is concerned, Vishwac Sena Kannan showed an interesting slide in his session **[Let's talk about Conversation Design](https://myignite.microsoft.com/videos/55282)** about the growth and usage of this awesome framework which you can see below: {{< figure src="/images/bots_growth-1024x537.png" >}} ### New Capabilities * Azure Machine Learning: 2 new services (**[Experimentation & Model Management Service](https://azure.microsoft.com/en-us/blog/diving-deep-into-what-s-new-with-azure-machine-learning/)**) * Azure Machine Learning: New Tools (**[Visual Studio Code Tools for AI & Azure Machine Learning Workbench](https://azure.microsoft.com/en-us/blog/diving-deep-into-what-s-new-with-azure-machine-learning/)**) * Azure Machine Learning: **[AI powered data wrangling](https://azure.microsoft.com/en-us/blog/diving-deep-into-what-s-new-with-azure-machine-learning/)** * Azure Machine Learning: Deploy models to the **Edge** (e.g.: **IoT** solutions) * Azuer Machine Learning: **Layered** & **hierarchical** **model** **deployment** possible * SQL Server 2017: Support for **R** and **Python** * * * ## 27.09.2017 Unfortunately, I have not spotted any roadmap updates for all of these services, but I would take this opportunity to list some really cool sessions you should watch on demand if you are not at Ignite like me: * **[Language with Microsoft Cognitive Services: infusing language and speech capabilities into your apps](https://myignite.microsoft.com/videos/55283)** * [**Microsoft Cognitive Services - Infuse intelligence into your business app with Vision and Speech**](https://myignite.microsoft.com/videos/55284) * [**Let’s talk about Conversation Design**](https://myignite.microsoft.com/videos/55282) * **[Bot Framework patterns and practices straight from our customer](https://myignite.microsoft.com/videos/55288)** The last session about Bot Framework patterns and practices gives a nice overview of the lessons learned which you can see below: {{< figure src="/images/capture20170928075414109-1024x402.png" >}} --- title: "Cognitive Services - 101 (with Infographic)" url: https://bisser.io/cognitive-services-101-with-infographic/ markdown: https://bisser.io/cognitive-services-101-with-infographic/index.md date: 2017-09-14 lastmod: 2017-09-14 tags: AI, Bot Framework categories: AI, Cognitive Services description: "Cognitive Services - 101 (with Infographic)" status: legacy reading_time_min: 4 --- **[Microsoft Cognitive Services](https://azure.microsoft.com/en-us/services/cognitive-services)** are APIs and services which can be used to add intelligent services to applications for different purposes, whether this is a **web**, **mobile** or **desktop app** or a **bot** which should interact with humans in a more natural way. But as there is a wide range of **Cognitive Services** which can be used this post should expose and describe each of the APIs currently available. This should help get to know **Microsoft's AI ecosystem** a bit more. Additionally, the infographic I came up with should give an overview of the Cognitive Service world, which can be used to describe the individual services and APIs to help understand which service should be used in which area of application. * * * > Microsoft Cognitive Services let you build apps with powerful algorithms to see, hear, speak, understand and interpret our needs using natural methods of communication, with just a few lines of code. Easily add intelligent features – such as emotion and sentiment detection, vision and speech recognition, language understanding, knowledge, and search – into your app, across devices and platforms such as iOS, Android, and Windows, keep improving and are easy to set up. [Read more…](https://azure.microsoft.com/en-us/services/cognitive-services) * * * Looking at the following infographic, the **Cognitive Services** can be grouped into six main areas, which are explained in detail below: * Vision * Knowledge * Speech * Language * Search * Labs ![Cognitive-Services-Infographic-with-Description](/static/img/Cognitive-Services-Infographic-with-Description-new-725x1024.png) Cognitive-Services-Infographic-with-Description ## Vision With the different vision APIs, it is possible to gain insights from **pictures** or **videos**. The capabilities range from **detecting faces** in pictures up to **automated image, text or video moderation** as well as **person recognition**. As the infographic shows, there is even the possibility of building an own computer vision model for custom use cases, if the other services are not sufficient for the goal which should be achieved. ## Knowledge Whether you want to build a **QnA** **Bot** with the QnA Maker API or you want to build an app which serves the users with recommendations based on past data, this category offers some nice services, which can be used to implement knowledge features into your apps. Additionally, what's quite cool is the **Custom Decision Service** which lets you build a**contextual decision-making service** which is self-learning with experience. ## Speech As speech is the new and modern way to interact with a service, the speech APIs give you the possibility of implementing**speech translation** or **speaker recognition** features into your apps to make it more human. With the Bing Speech API, you can even **convert text to speech** and vice versa on the go to understand your users' intent. ## Language With the introduction of my **Azure Bot ["Homie"](https://bisser.io/homie-a-language-understanding-bot-for-smart-homes/)**, I already showed you how to use one of the services in the language category: **Language Understanding Intelligent Service (LUIS)** in order to understand the intent of the user who interacts with my Bot. Furthermore, the other APIs can be used to **detect the sentiment** of a given text (like I did with **[Survey 365](https://bisser.io/survey-365-an-intelligent-survey-solution/)**) or**translate text** into other languages. ## Search Search is one of the most important services for nearly every application or solution nowadays. In order to implement a search service, it is essential to provide the best possible results. Therefore the services and APIs in the search category are all offered by **Bing** and can be used to search the content of different types of **news, entities, images or videos**. In addition, the **Bing Custom Search** can be used to implement custom search routines to deliver the results you want your users to find. ## Labs As the Cognitive Service APIs are rapidly growing the labs can be used to evaluate possible future APIs which are currently development by Microsoft. The current spectrum is ranging from **gesture based controls** to **route logistics** and **location insights**. ## Conclusion Microsoft Cognitive Services offer powerful and easy to use APIs for adding intelligence to various apps and solutions without the need to hosting the services on your own. The main benefits are: * Easy to use services * **NodeJS** and **C#** support in terms of development * Can be implemented nearly everywhere (web apps, desktop/mobile apps, bots, … * Powerful services with a wide range of possible use cases * * * The full image can be downloaded here: **[Cognitive-Services-Infographic-with-Description](https://github.com/cloudguy-pro/Infographics/blob/master/Cognitive-Services-Infographic-with-Description-new.png)** Feel free to use it for internal use in your presentations or documents. The only point I ask you to do is to credit me and link this article in your documents where you use it. Please do not sell this as your own ;) For PowerPoint slides I have also created a more handy infographic, which should fit better on slides (it actually doesn't show the descriptions of each API). You can download it here:** [Cognitive-Services-Infographic](https://github.com/cloudguy-pro/Infographics/blob/master/Cognitive-Services-Infographic.png)** ![Cognitive-Services-Infographic](/static/img/Cognitive-Services-Infographic-2-768x798.png) Cognitive-Services-Infographic --- title: "Homie - A language understanding bot for smart homes" url: https://bisser.io/homie-a-language-understanding-bot-for-smart-homes/ markdown: https://bisser.io/homie-a-language-understanding-bot-for-smart-homes/index.md date: 2017-08-16 lastmod: 2017-08-16 tags: AI, Bot Framework categories: AI, Bot Framework description: "Homie - A language understanding bot for smart homes" status: legacy reading_time_min: 8 --- Smart homes, smart vehicles, smart everything - nowadays all devices which we daily use have to be smart otherwise we wouldn't use them, would we? There are so many different approaches to make a device smart either you can control it remotely, speak to it or it even sets actions autonomously which should ease our lifes here and there. The key to achieve all this is nearly the same everywhere, no matter if you use smart devices and services provided by Google, Amazon or Microsoft. Behind all that is always**AI** (**Artifical Intelligence**). And as I do a lot of research on Microsoft's AI stack (**Cognitive Services**, **Bot Framework**, **Machine Learning**, ...) within **Azure**, I wanted to develop something intelligent, which could make my everyday life a bit easier. So I thought of a service, which could take actions whenever I tell it to do so. And this was the birth of **"Homie"**, who is my personal assistant for doing smart stuff. But now you most probably want to know who or what Homie is, right? Well, I'll try to explain you what and who he is within the next few paragraphs. #### The Solution Design Let's start. Imagine you are sitting in your office at home or at work and are really deepened in your work and you have to be concentrated in order to get your work done. And as time flies by, it gets dark outside. In the beginning you won't notice that it gets dark as you only star into your monitors and do your work. But after a few minutes, when it's really dark, you are sitting in a dark room and we all know, this doesn't work out very well for a longer time. But as you are a lazy person, like me, you don't want to stand up and go over to the light switch to turn the light on. But what if I told you that Homie could turn the light on for you and you can remain seated? All you need to do is to follow the following steps and get all services connected to each other like in the picture below and you can tell Homie what to do for you ;) {{< figure src="/images/homie_overview-1024x704.png" >}} The design of the given solution as well as the flow of communication is shown above #### The Bot and Cognitive Services Stuff First of all we need to create an Azure Bot service, which is responsible for communicating with the user. So head over to the Azure portal and create a new bot. In my case I created a NodeJS LUIS bot, as there is already some LUIS related stuff preconfigured. Now when the bot is ready you need to add some intents to your LUIS app in order for the bot to actually be smart. So go to [luis.ai](https://www.luis.ai) and login (or register if you did not already) and add the following pre-defined intents: {{< figure src="/images/luis-2-300x177.png" >}} These are the pre-defined LUIS intents used in this Cognitive Services app Now test and train your LUIS app and go back to your bot in the Azure portal, as we now will insert some magic. Now the following piece of code, let's your bot become smart as this code is responsible for doing actions based on the detected intent of a received messages. {{< highlight js >}} intents.matches('HomeAutomation.TurnOn',function(session, args){ session.sendTyping(); if(args.entities[0]) { var deviceEntity = builder.EntityRecognizer.findEntity(args.entities, 'HomeAutomation.Device'); var roomEntity = builder.EntityRecognizer.findEntity(args.entities, 'HomeAutomation.Room'); if (roomEntity){ session.send("Ok, I will turn the %s on in the %s", deviceEntity['entity'], roomEntity['entity']); session.endDialog(); var commandJSON= { "device": deviceEntity['entity'], "room": roomEntity['entity'], "action": "Turn on" }; request({ url: "", method: "POST", json: true, // <-- This is very important! body: commandJSON }, function (error, response, body){ console.log(response); }); } } }); {{< / highlight >}} #### The IoT Stuff Now that your bot is ready to receive messages, we need to create an Azure IoT Hub which is used to send and receive messages from and to devices. So create one in your portal and add your devices to the hub according to your needs. In this example I used a device for my office, which I called "office_controller". Make sure to note the keys of your device, as you will need them later on: {{< figure src="/images/IoT_device-263x300.png" >}} There are only a few clicks needed to create a new IoT device in the IoT Hub blade #### The Azure Functions Stuff Now that we have our bot and the IoT Hub up and running, we need to connect those services with each other. And I thought of doing so by using a JavaScript HTTP trigger function. So create a function in the Functions portal and make sure to install the following dependencies by using the command line within your function's settings: {{< highlight Shell >}} npm install azure-iothub --save {{< / highlight >}} Now to reference your created IoT Hub in the function's code you can use the following piece of code: {{< highlight javascript >}} var Client = require('azure-iothub').Client; var Message = require('azure-iot-common').Message; var connectionString = ''; var targetDevice = 'office_controller'; var serviceClient = Client.fromConnectionString(connectionString) {{< / highlight >}} In order to send a message to your IoT Hub, use the following code snippet: {{< highlight javascript >}} var message = new Message('lightsOnOffice'); message.ack = 'full'; message.messageId = "ID_001"; serviceClient.send(targetDevice, message, printResultFor('send')) {{< / highlight >}} #### The Device Stuff Now this is the last piece of work to do in order to actually make your hard work visible. In this case I used a Raspberry Pi 3 in combination of a breadboard and a normal LED light, as I don't have a smart light bulb at hand, which I could have used instead. But I guess this is ok, as I want to show you that the solution is working rather than the light bulb is IoT enabled. So now it's time to assemble your PI in order to get the LED connected to it. Below you'll find the actual construction manual by Microsoft, which I followed to connect the LED on the breadboard with my Pi via GPIO. {{< figure src="/images/pinout_breadboard-300x228.png" >}} This is the pinout manual on the Raspberry PI's breadboard for connecting a LED light taken from [GitHub](https://github.com/Azure-Samples/iot-hub-node-raspberrypi-getting-started/blob/master/doc/iot-hub-raspberry-pi-kit-node-lesson1-configure-your-device.md) When I finished that, I installed NPM and Node JS on my PI in order to run a little Node JS app, which is actually listening on the IoT hub's device commands. Whenever there is a new command in the queue for this specific device, a function gets triggered, which does something. In my case I wanted to turn the lights on or off in a specific room. The actual code for lightening up your LED is something like this: {{< highlight javascript >}} var wpi = require('wiring-pi'); // GPIO pin of the led var configPin = 7; wpi.setup('wpi'); wpi.pinMode(configPin, wpi.OUTPUT); // Turn LED on var isLedOn = 0; isLedOn = +!isLedOn; wpi.digitalWrite(configPin, isLedOn ) {{< / highlight >}} And that's basically it. Now it's time to tell Homie to turn the light on and off in the office and see if it's working. Take a look at the [video](https://www.youtube.com/watch?v=kZPuI3JwVww) to find out the answer. And yes you're right, this is only a LED light, which Homie turned on and off, but the process is actually the same and I wanted to proof that the given problem can be solved by a bot. But please feel free to adapt my solution and add real light bulbs or other things to it in order to make it even more realistic... #### The Teams Stuff Now that your bot is fully functional and doing what you want him to do, it's time to add the desired channels to that bot in order to publish it into your favourite apps. I decided to go for Microsoft Teams, as this is one of my favourite communication tools. But you can choose whatever you like from the following list of channels: {{< figure src="/images/teams_channels-300x103.png" >}} These are the available channels, in which the bot can be added After you have added the Teams channel you can communicate with the bot via Teams, just as simple as that: {{< figure src="/images/homie_teams_communication-300x151.png" >}} The communication with your bot within Teams is like communicating with your co-workers #### The Conclusion * **Azure**, **Cognitive Services** and the **Bot Framework** fit perfectly together, as the interaction between the services is working out of the box * **Azure Functions** are always a good use for hosting logic and code which is consumed by other services * **Language Understanding Bots** will likely to be introduced more and more in consumer as well as business solutions as they make life easier a lot * With less effort,**Cognitive Services** can be used to make a service more human and lets therefore interact people with a service like a bot in a more natural way, which is quite promising Now that this solution is working, I'll update you with more features concerning this scenario as well as publishing the code and other stuff which was used in this solution as an open-source solution so everyone of you can set up a personal **Homie** as well. If you have any questions, notes or any other use cases for this bot (or other use cases), which may be interesting, please let me know down below or via any social media channel as I would like to hear your feedback on this one... By the way, this blog post was also released on the official MSDN Team Blog Austria in German if you are interested take a look at [MSDN Team Blog Austria (Codefest)](https://blogs.msdn.microsoft.com/codefest/2017/09/07/homie-ein-bot-der-die-menschliche-sprache-versteht/) --- title: "Survey 365 - An intelligent survey solution" url: https://bisser.io/survey-365-an-intelligent-survey-solution/ markdown: https://bisser.io/survey-365-an-intelligent-survey-solution/index.md date: 2017-07-17 lastmod: 2017-07-17 categories: AI, Cognitive Services description: "Survey 365 - An intelligent survey solution" status: legacy reading_time_min: 5 --- **Microsoft Forms** is one of the youngest services in the **Office 365** ecosystem. Forms is a great service in my opinion as such a service was missing in the Microsoft repertoire. Now Office 365 users can create their Forms within seconds in order to create a survey or quiz and ask others about their opinion. And one of the most used survey topics in a company is the wellbeing of the employees. If the employees do not feel well, they won't achieve the necessary results, which would push the companies services forward. And we all know what happens if the employees of a company don't like what they're doing in their every day's work - either they quit soon or they quit later, but they'll for sure quit if nothing else happens. But the time, where they don't perform as expected is a crucial part, which needs be as low as possible. But how does this belong to Microsoft's new Forms service? Well, I will tell you. How about if you could create a survey for your employees in order to get to know their opinion about their job position and their performance? How about if your employees would really tell you what they feel (as we all know they won't, or at least 90% won't in a face to face discussion as they would fear the results)? My pitch on this would be to create a survey for that which could be sent out to the employees on a regular basis in order to gain insights on their current situation. But why would we use Forms for that? Well, that's an eligible question, but my answer would be: Why shouldn't we use a Microsoft, or even better an Office 365 service for that as we love this ecosystem? But there's an even better answer to this - Because we can use other great tools along with Forms which offer us whole new opportunities. And therefore I want to show you how you can create an intelligent survey based on Forms, Teams and of course AI in order to create a real intelligent survey, which uses **Cognitive Services** for sentiment detection of crucial answers and alerts the HR team via **Microsoft Teams**, if an employee gives a negative answer to a question. This makes it possible to filter out good and not so good feeling employees easily in order to set actions based on the answers real quick. This may lead to a very small delta, in which employees don't like what they're doing before they'll like it again. ## The Solution Overview {{< figure src="/images/survey365_01.png" >}} ## Creating our first Form First of all we'll create a new form by browsing to [https://forms.office.com/Pages/DesignPage.aspx](https://forms.office.com/Pages/DesignPage.aspx) or launching the service via the O365 app launcher. Now let's choose "New Form" and start inserting your questions. My Form for this scenario looks like the following, although I have to admit that there are certainly better formulated questions for asking employees how they feel, but I'm not a specialist in that field ;) {{< figure src="/images/survey365_13.png" >}} ## The Glue for our Services - Microsoft Flow Now when we are finished with our Form, we'll create a new **Flow** (but you can also create a new **Azure Logic App** btw) and search for the "Forms" connector and select the trigger "Microsoft Forms - When a new response is submitted": {{< figure src="/images/survey365_02.png" >}} As everything in the cloud is connected, we can just select our Form, which we created earlier: {{< figure src="/images/survey365_03.png" >}} Now we need to add a new action which will be the sentiment detection powered by the Azure Text Analytics API. The text which i will check is the answer to the question "How do you feel at work?" but as you might guess we could check more than one as well in order to get better and more precise results. {{< figure src="/images/survey365_04.png" >}} In order to alert the Teams channel if a negative answer was submitted, we need to insert an action which is called "Condition" and we will enter the following condition settings in order to be able to alert the Team only if the sentiment score of the given answer is less than 0,5 (you can also adapt that value): {{< figure src="/images/survey365_05-1.png" >}} And now we have the ability to trigger an action if the condition is true (yes). So we want to post a message to our Teams channel with the following message based on the user information and sentiment score: {{< figure src="/images/survey365_06.png" >}} Lastly, we also want to save the answers and information in a SharePoint Online list for further use and searchability (as searching content in Forms is not that well implemented yet): {{< figure src="/images/survey365_07.png" >}} Now we need to save the Flow, which should look like this: {{< figure src="/images/survey365_08.png" >}} ## The First Responses on our Form Finally we are going to test our intelligent survey. Now let's answer the Form two times: First we will answer it as a very happy employe who loves his job every single day: {{< figure src="/images/survey365_09.png" >}} Secondly we'll answer it as a employe who doesn't like his job and wants to do something else better today as tomorrow. {{< figure src="/images/survey365_10.png" >}} ## What about the Sentiment? Now let's check out the SharePoint Online list first to see if we have any results in there: {{< figure src="/images/survey365_11.png" >}} And yes, we have two entries in our SPO list. But that's not the best part of it. Checking out the Teams channel we'll see that there has been a negative answer submitted by a user: {{< figure src="/images/survey365_12.png" >}} And that's pretty cool how we can set up an intelligent survey solution which uses our favourite tools and service in order to analyze the answers and set conditional actions if the sentiment of an answer is lower than a defined threshold and post a message to a Teams channel. Of course this is is only one of many use cases for this solution as you can also do this in customer scenarios or others. But this solution should show how powerful Teams and Forms are together with the support of AI and Office 365… ## Conclusion * Forms is an easy to use service within Office 365 for creating surveys, questionnaires and quizzes * Sharing with internal and external users can be done via E-Mail, Links, QR-Codes and Embed Codes * Cognitive Services are a nice way to bring Intelligence into various services without the need to write actual code --- title: "How to keep up with the Azure roadmap" url: https://bisser.io/how-to-keep-up-with-the-azure-roadmap/ markdown: https://bisser.io/how-to-keep-up-with-the-azure-roadmap/index.md date: 2017-07-11 lastmod: 2017-07-11 categories: Azure description: "How to keep up with the Azure roadmap" status: legacy reading_time_min: 2 --- Many of my customers, where I had the chance to do a workshop about Azure or Office 365 asked me the same question every single time: "How do you keep track with the updates on the roadmap?" I have to admit, the question is eligible due to the fact, that there is an update on the roadmap for both products nearly every day. So as a good consultant, you would need to spend all of your time studying the roadmaps to be on track and give your customers the newest information and advices in order to sell the best services and solutions. Well, I guess we all know, that this can't be done, except for people who don't sleep. So i thought about a better way of how to keep track with the updates on the different roadmaps. Looking at the following 2 pages, where the roadmaps of Office 365 and Azure are displayed, I soon detected something on the Azure roadmap page, which I could use - RSS feeds. So take a look at the roadmaps if you like: * [https://azure.microsoft.com/en-us/roadmap/](https://azure.microsoft.com/en-us/roadmap/) * [https://products.office.com/en-us/business/office-365-roadmap](https://products.office.com/en-us/business/office-365-roadmap) So I thought of a solution which should give me the possibility to stay on track, no matter where I am and which device I am using. That's an ideal use case for Microsoft Teams. So basically what I have done is to built a Flow, which should be triggered if a RSS feed item is published and afterwards send the feed item summary to my Teams channel for later reading. So let's head over to Flow and create a new one with the following trigger settings: ![](/static/img/roadmap_01.png) The RSS feed URL is the Azure roadmap URL followed by "/feed" (this can be found on the roadmap page as well). So paste it in there: ![](/static/img/roadmap_02.png) Now we only need to add an action which should post a message to my Teams channel, like this: ![](/static/img/roadmap_03.png) And if you saved it, you will soon see some new messages in your channel, which inform you about the latest Azure roadmap updates: ![](/static/img/roadmap_04.png) Unfortunately, I did not find anything similar for Office 365 as well, as there is no official RSS feed for this roadmap. But I'm pretty sure, they will add this to the O365 roadmap as well. So if this is there, I will update you on how to combine both of them with Teams in order to built up a roadmap update service… --- title: "Detect the sentiment of your received emails" url: https://bisser.io/detect-the-sentiment-of-your-received-emails/ markdown: https://bisser.io/detect-the-sentiment-of-your-received-emails/index.md date: 2017-06-29 lastmod: 2017-06-29 categories: AI, Cognitive Services description: "Detect the sentiment of your received emails" status: legacy reading_time_min: 5 --- The last days I was doing a lot of researches on AI and Cogntive Services as this topic is a major topic for the future in the daily use. So I read a couple of blog posts about this and got an idea on how to use it. And today I want to show you a small prototype application which I created in order to combine the Azure Cognitive Service API with my most used tool for work, which is Outlook and packed it into an Azure Logic App as I am not a developer (as you might know by now) in order to save development effort. The basic idea was to create a solution which checks my received mails and detects the sentiment of the mail body in order to see if the sentiment score is high or low of the received mail. Therefore I created a logic app which is consisting of the following actions: ![](/static/img/capture20170629105229587.png) The Azure Logic App triggers each time a new mail arrives in my inbox, sends the body to the Text Analytics Service of the Cognitive Service API and the result which is basically the sentiment score of the mail will be inserted into a Power BI dataset in order to build a nice dashboard. But let's start from the beginning and create the Logic App. So head over to your Azure portal and create a new Logic App: ![](/static/img/capture20170629110231049.png) When the app is deployed you can head over to the Logic App Designer and start to insert the first action which is the Outlook action "When a new email arrives". So if you open up the Designer you will be presented with a couple of templates. But we will go with the blank one: ![](/static/img/capture20170629112000242.png) Next we will search for our trigger in the Office 365 Outlook service and click on "When a new email arrives": ![](/static/img/capture20170629112100431.png) Now the Designer is prompting us to sign into your Office 365 mailbox so we will do that: ![](/static/img/capture20170629112215482.png) If you have SSO enabled, you won't need to enter your credentials here. When you signed in you can choose your mailbox folder which should be checked. I will go for the Inbox in my case but you can choose whatever folder you want. You can also adapt the frequency the trigger should be fired. My settings are look like this: ![](/static/img/capture20170629112356964.png) Now we will add a new step which is the detection of the sentiment of the mail body. But before we can do this we need to create the Text Analytics service first. So save your app and click on the "+" symbol in your navigation pane of the Azure portal and search for "Cognitive Services" then click create. Now choose the following settings and change the Name, subscription and pricing tier to fit your needs (I have chosen the S1 priecing tier but you are good to go with the free one which is F0 as well!): ![](/static/img/capture20170629113033705.png) Now that this is created you can go to that service and get the necessary key: ![](/static/img/capture20170629113152165-1.png) Now we can go back to our Logic App and go to the designer to add a new action. To do this click "+ Next step" - "Add an action": ![](/static/img/capture20170629112459089.png) Now we want to search for the connector "Text analytics" and choose the action of "Detect sentiment". Now it asks us for a name and a key: ![](/static/img/capture20170629112621764.png) So we enter the name of the Text Analytics service we created earlier and the key. Now we need to enter the text which should be analysed, so we choose the Body of the email which we will receive: ![](/static/img/capture20170629113502759.png) So now we need to configure Power BI in order to receive the necessary information for our dashboard. So go to [https://app.powerbi.com](https://app.powerbi.com) and create a new streaming data set in your workspace as follows: ![](/static/img/capture20170629121922437.png) Choose API as the source of your data: ![](/static/img/capture20170629121934795.png) Give it a name and insert values for the incoming data: ![](/static/img/capture20170629122046052.png) As we want to show the sender's email address and the sentimentscore on our dashboard we need these two values. Additionally we will add the subject as a value. It is very important to enable "Historic data analysis" in order to show the values in the dashboard. When you're ready click create. Now we can go back to our Logic App Designer and add another action. This time we will add the "Power BI - Add rows to a dataset" action.  Now we need to sign into our Power BI profile: ![](/static/img/capture20170629122943457.png) After we have signed in we need to provide the Workspace, Dataset (which we have just created) and the table information. If you have entered the data correctly you will be asked for our data values, which we defined in Power BI - subject, sentiment score and the sender: ![](/static/img/capture20170629123214407.png) After entering the data we just need to save the app and we are good to go to test it. The full design of the app should look like this: ![](/static/img/capture20170629124044478.png) So click on the run button in the Designer and send an email to your email address with a nice text. After a couple of seconds you should see that your app was triggered: ![](/static/img/capture20170629123536315.png) This means, that you can go on building your dashboard in Power BI and add the necessary data to a report which you can then link to a dashboard. Mine looks like this (very clear and not with much fancyness): ![](/static/img/capture20170629123812730.png) So now you know how to implement a sentiment detection on received emails and insert the gathered data into a Power BI dashboard for analytics of the senders who like you and the ones who don''t ;) Stay tuned as I will further develop this solution and add more functionality into it. Cheers! --- title: "ADFS 2016 idpinitiatedsignon page is missing - Solved" url: https://bisser.io/archive/019-adfs-2016-idpinitiatedsignon-page-is-missing---solved/ markdown: https://bisser.io/archive/019-adfs-2016-idpinitiatedsignon-page-is-missing---solved/index.md date: 2017-06-28 lastmod: 2017-06-28 status: legacy reading_time_min: 1 --- Many of you guys will probably know the ADFS page, which can be used to test the authentication. Most of the time this page is used after the installation of an ADFS farm or in error cases. With the introduction of ADFS on Windows Server 2016, many of us may have encountered the problem, which is shown in the following screenshot when browsing to the URL [https://adfs.corp.com/adfs/ls/idpinitiatedsignon](https://adfs.corp.com/adfs/ls/idpinitiatedsignon) after the successful installation of a new ADFS farm: ![](/static/img/adfs2016_idp_01.png) Ooops something went wrong. But is my ADFS farm installation missing a piece? Well, kind of! If you encounter this, then run the following PS command: ```powershell Get-AdfsProperties | select EnableIdPInitiatedSignonPage ``` Most probably you will see this result: ![](/static/img/adfs2016_idp_02.png) So the problem is that this page is disabled by default. So let's change this by running the following command: ```powershell Set-AdfsProperties -EnableIdPInitiatedSignonPage $true ``` ![](/static/img/adfs2016_idp_03.png) Now you can browse to your InitiatedSignon page and you will be able to login and check if everything is working as expected! Happy ADFSing ;) ![](/static/img/adfs2016_idp_04.png) ![](/static/img/adfs2016_idp_05.png) --- title: "Microsoft Teams ♥ Microsoft Stream" url: https://bisser.io/archive/018-microsoft-teams--microsoft-stream/ markdown: https://bisser.io/archive/018-microsoft-teams--microsoft-stream/index.md date: 2017-06-21 lastmod: 2017-06-21 status: legacy reading_time_min: 2 --- With the GA announcement of Microsoft Stream yesterday, I thought of using it in a very cool and modern way. My idea was to create an interactive video platform within my Teams service. So let's hit it and configure Stream and Teams in a way to watch Stream videos within your Teams client and build up a modern video streaming platform. First of all you need to create a new Stream channel. So I went over to my Stream app in the app launcher of my O365 tenant an created a new channel there, like the following: ![](/static/img/capture20170621103454888.png) You can enter whatever you like. When you're done hit create and wait for the channel to be created. After that you can start uploading your favourite videos. I chose some Azure Friday episodes as these are quite interesting for me. ![](/static/img/capture20170621104014222.png) So when you're done uploading videos you can go over to your Teams client. I created a new channel in my team which should be used for the videos. After that I added a new tab in that channel and select the Stream service: ![](/static/img/capture20170621104101893.png) So you will need to enter a name for that tab and the URL of your stream channel: ![](/static/img/capture20170621104132651.png) When you save that, you will see all of your uploaded videos from the Stream channel: ![](/static/img/capture20170621104207444.png) Now you are done configuring the client and you can grab your popcorn bag and start watching videos within your Teams client, isn't that a cool thing? ![](/static/img/capture20170621104302779.png) Happy watching! ;) --- title: "Solving the problem of the MyApps mobile app on iOS \u0026 Android not authenticating successfully" url: https://bisser.io/archive/017-solving-the-problem-of-the-myapps-mobile-app-on-ios--android-not-authenticating-successfully/ markdown: https://bisser.io/archive/017-solving-the-problem-of-the-myapps-mobile-app-on-ios--android-not-authenticating-successfully/index.md date: 2017-06-19 lastmod: 2017-06-19 status: legacy reading_time_min: 2 --- Today I want to bring some light into the darkness and write down, what's not written down as per today. I guess most of you know the MyApps portal, which is also feeding the Office 365 app launcher with enterprise applications. You can browse to that app panel directly via [https://myapps.microsoft.com](https://myapps.microsoft.com) and it is a very useful service, which should be used in every cloud environment as it provides SSO to apps without the need of remembering credentials for those apps like the corporate Twitter acocount or the corporate Amazon account. And the cool thing about this is that there is also a mobile app available on iOS and Android. But there seems to be hidden information, which is very important in my opinion, but is nowhere documented. We wanted to play around with the MyApps portal and added some applications through the Azure AD and did some testing on group based assignements to those apps, which perfectly worked fine in the browser. But there was this annoying problem, that all of this was not working on our iPhones and Androids via the MyApps mobile app. So i digged deeper into the Azure AD portal to see if there is some setting missing - nothing. I digged deeper on the ADFS if there might be some setting missing - no success. At that point we only used 1 ADFS server and no WAP. At the same time we deployed a test/dev environment which was kind of similar to our productive environment. As I did the same tests with the MyApps app there it worked out perfectly fine on my iPhone. So I thought of the differences between those two environments - and there was only one thing different: in the test environment we had a WAP in place for handling external authentication requests to the ADFS. So I decided to deploy a WAP in our production environment as well - et voila - suddenly the MyApps mobile app worked like charm here as well. So concluding this, it seems that the MyApps mobile app needs a WAP in place in order to successfully authenticating against the ADFS service. If this was documented somewhere, I could have saved some time worrying about this annoying problem, but I hope that this helps some of you out there overcome this issue… --- title: "Move Azure resource to another subscription with the Azure Cloud Shell" url: https://bisser.io/archive/016-move-azure-resource-to-another-subscription-with-the-azure-cloud-shell/ markdown: https://bisser.io/archive/016-move-azure-resource-to-another-subscription-with-the-azure-cloud-shell/index.md date: 2017-06-08 lastmod: 2017-06-08 status: legacy reading_time_min: 2 --- A couple of days ago I wanted to tidy up my Azure subscription and move the unneeded resources from my active subscription to my team's subscription. At first I wanted to move a storage account, which I created a couple of years ago - needless to say that this was of type classic resource. The Azure portal now introduced a quite handy operation which can be used for many resources in order to move the resource either to another resource group or even to another subscription. Yeah I know what you think right now - AWESOME. ![](/static/img/move_to_arm01.png) So as I hit the "Move" button in the Azure portal in order to move the account to another subscription I got the following error: ![](/static/img/move_to_arm02.png) And there it was - the problem of classic resource type migrations. So I searched a bit and had the idea of doing all needed operations within the Azure Cloud Shell. So let's start off by opening the shell and registering a migration resource provider and set up the preparations for the migration of the storage account. ```shell azure provider register Microsoft.ClassicInfrastructureMigrate azure provider show Microsoft.ClassicInfrastructureMigrate azure config mode arm azure storage account prepare-migration ``` After runnning the last command you should see something like this in the Azure Cloud Shell: ![](/static/img/move_to_arm03.png) So now let's do the migration by entering the following command which should lead to the following output: ```shell azure storage account commit-migration ``` ![](/static/img/move_to_arm04.png) So we have migrated the storage account from type classic to resource manager type. And now we are good to go to move the storage account to another subscription. And as we already have our shell session open let's do this via the cloud shell as well. Just enter the following command and make a note of the ID: ```shell az resource show -g -n --resource-type "Microsoft.Storage/storageAccounts" --query id ``` ![](/static/img/move_to_arm05.png) And now finally we can move the resource to the destination subscription with the following command: ```shell az resource move --destination-group --ids --destination-subscription-id ``` ![](/static/img/move_to_arm06.png) This command will take a while to complete, but if you watch the destination resource group you will likely see the following message "Moving resources" as the following illustration shows: ![](/static/img/move_to_arm07.png) And when the command is completed successfully you have moved your storage account. Of course you are not limited to storage accounts only, but you can move almost any service in the ARM world between resource groups or subscription. If you want to know how to move another resource type, just give me short notice below or on Twitter and I will follow up on that! Cheers… --- title: "How to create an Office365 alert in Microsoft Teams in less than 10 minutes" url: https://bisser.io/archive/015-how-to-create-an-office365-alert-in-microsoft-teams-in-less-than-10-minutes/ markdown: https://bisser.io/archive/015-how-to-create-an-office365-alert-in-microsoft-teams-in-less-than-10-minutes/index.md date: 2017-05-29 lastmod: 2017-05-29 status: legacy reading_time_min: 2 --- As I played around with Microsoft Flow and Teams I had the idea of creating a channel with news on Office 365\. So I created a Teams channel and started creating a Flow, which was pretty easy. And as I was finished and I started the flow, the Teams channel which should show me tweets with the hashtag #Office365, soon was overfilled with tweets about Office 365 in all kinds of languages. So I thought of creating a Flow, which should filter out all other languages than English, as most useful tweets are posted in English (but as you might guess you can adapt my solution and use any language, which supported by Azure Cognitive Services). Therefore, I created a new Azure Cognitive Service in my Azure Subscription as the following graphic shows. ![](/static/img/Flow01.png) Now you need to note your Cognitive Service name and key from the Azure Portal and head over to the Flow portal, where you create a new Flow as follows. ![](/static/img/Flow02-1024x643.png) And this is basically it. Now hit Done and you are ready to look your Teams Channel. I tested it myself and i I have to admit although it's still in preview (the language detection) it works pretty good, take a look: ![](/static/img/Flow03-976x1024.png) I will further play around with these services in order to deliver an even better solution and I will try to combine new services soon, so stay tuned for updates… --- title: "A guide on simplifications for O365 admins - III - Visual Studio Code profiles" url: https://bisser.io/archive/014-a-guide-on-simplifications-for-o365-admins---iii---visual-studio-code-profiles/ markdown: https://bisser.io/archive/014-a-guide-on-simplifications-for-o365-admins---iii---visual-studio-code-profiles/index.md date: 2017-05-24 lastmod: 2017-05-24 status: legacy reading_time_min: 2 --- In the last post I tried to show how to make PowerShell life easier. You can read it [here](http://bisser.io/a-guide-on-simplifications-for-o365-admins-ii-powershell-profiles/) This post is picking up the same issue, but the tool is a bit different. All cloud admins may know that Visual Studio Code supports PowerShell scripts and in addition to that makes life easier by having a source control for those scripts via GitHub oder TFS. But still you may encounter the problem, that you have to login to every cloud service seperately in every VSCode session, which can be annoying. But as VSCode supports PowerShell you can apply the same set of functions mentioned in my previous post in VSCode as well. All you have to do is open up VSCode create a new PSScript and run the following code: ```powershell Test-Path $Profile ``` If you receive a False, then you can set up your profile with the following commands: ```powershellNew-Item –Path $Profile –Type File –Force Notepad $Profile ``` Now this opens up a blank profile, which needs to adapted in order to implement functions, which allow you to login to all cloud services, like we did in the PS DIE. Therefore, you can paste the following code: ```powershell### Import Modules Import-Module MSOnline function Connect-Tenant1 { [Cmdletbinding()] param ( [Microsoft.PowerShell.EditorServices.Extensions.EditorContext]$context ) $CloudUsername = "admin@tenant1.onmicrosoft.com" $File = "C:\Users\adm.sbi\Documents\Tools\PowerShell\tenant1_pw.txt" $CloudCred=New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $CloudUsername, (Get-Content $File | ConvertTo-SecureString) #Connect to Office 365 Connect-MsolService -Credential $CloudCred #EXO     $SessionEX = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $CloudCred -Authentication Basic -AllowRedirection Import-PSSession $SessionEX     Add-AzureAccount -Credential $CloudCred Select-AzureSubscription -SubscriptionName "AzureSubscription" Login-AzureRmAccount -Credential $CloudCred Import-Module "C:\Program Files\SharePoint Online Management Shell\Microsoft.Online.SharePoint.PowerShell\Microsoft.Online.SharePoint.PowerShell.dll" Connect-SPOService -Url https://tenant1-admin.sharepoint.com -credential $CloudCred Clear-Host } ``` Now hit save and restart VSCode. Now if you start typing connect-ten you will get the suggestions Connect-Tenant1, which is pretty cool, isn't it? ![](/static/img/VSCode.png) And this is it, now you're good to go with your new PS environment. If you have further questions, you can of course contact my via Twitter, mail or other services… --- title: "A guide on simplifications for O365 admins - II - PowerShell profiles" url: https://bisser.io/archive/013-a-guide-on-simplifications-for-o365-admins---ii---powershell-profiles/ markdown: https://bisser.io/archive/013-a-guide-on-simplifications-for-o365-admins---ii---powershell-profiles/index.md date: 2017-05-16 lastmod: 2017-05-16 status: legacy reading_time_min: 3 --- In my previous post, which you can read [here](http://bisser.io/a-guide-on-simplifications-for-o365-admins-part-i-chrome-profiles/) I showed you how you can simplify your life as an O365 admin, with Chrome profiles. With this post, I want to show you, how you can tune your PowerShell in order to establish PS profiles too, which can be used to automate common tasks, like login and connect to cloud services like Exchange Online, Office 365 or Azure. Therefore, if you haven't created a PS profile already, just run the following command in the PowerShell or, what I personally prefer, in the PS ISE. ```powershell Test-Path $Profile ``` ![](/static/img/ps_profile01.png) If you receive a "False", you do not have a profile already, so let's create one with the following command and then open it in the notepad: ```powershell New-Item –Path $Profile –Type File –Force Notepad $Profile ``` ![](/static/img/ps_profile02.png) So now that you own a PS or ISE profile, you can do some nice stuff with it. And as an O365 admin, you probably love to do admin tasks with your PS instead with the GUI. But isn't it annoying to login in everytime you open up your PS or ISE or even worse, login with different credentials, when you need to administer a customers tenant? That's why I implemented my own login process in my ISE in order to define quick actions, which do the authentication process automatically with a few clicks. Simply paste the following code to your profile and replace with your credentials. ```powershell $psISE.CurrentPowerShellTab.AddOnsMenu.SubMenus.Add(     "Tenant1", {      $CloudUsername = 'admin@tenant1.onmicrosoft.com' $CloudPassword = ConvertTo-SecureString 'Pa$w0rd' -AsPlainText -Force Clear-Host $CloudCreds = New-Object System.Management.Automation.PSCredential $CloudUsername, $CloudPassword Connect-MsolService -Credential $CloudCreds      $SessionEX = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $CloudCreds -Authentication Basic -AllowRedirection Import-PSSession $SessionEX      Add-AzureAccount -Credential $CloudCreds Select-AzureSubscription -SubscriptionName "AzureSubscription" Login-AzureRmAccount -Credential $CloudCreds Import-Module "C:\Program Files\SharePoint Online Management Shell\Microsoft.Online.SharePoint.PowerShell\Microsoft.Online.SharePoint.PowerShell.dll" Connect-SPOService -Url https://tenant1-admin.sharepoint.com -credential $CloudCreds         Clear-Host     },     "Control+Alt+1" ) ``` Now if you re-open up your ISE (if you have followed the previous steps in your ISE) you should see the following Add-Ons options: ![](/static/img/ps_profile03-480x502.png) So now you are good to go and click on your Tenant and start using your PS profile. And as you might guess you can paste more of these Add-Ons into your profile in order to add more tenants. But what's not that ideal is that the password is inserted in plain text. Therefore you can follow the following routine in order to use encrypted passwords for your PS profiles. At first, run the following command in your PS in order to create an encrypted password and save it to your disk (you may adapt the path): ```powershell (Get-Credential).Password | ConvertFrom-SecureString | Out-File "C:\Users\adm.sbi\Documents\Tools\PowerShell\tenant1_pw.txt" ``` ![](/static/img/ps_profile04-768x213.png) Now that you have that in place, you need to adapt your profile in order to read the file, with the encrypted password and create your credential object with that password: ```powershell $psISE.CurrentPowerShellTab.AddOnsMenu.SubMenus.Add(     "Tenant1", { $CloudUsername = 'admin@tenant1.onmicrosoft.com' $File = "C:\Users\adm.sbi\Documents\Tools\PowerShell\tenant1_pw.txt" $CloudCreds = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $CloudUsername, (Get-Content $File | ConvertTo-SecureString) Connect-MsolService -Credential $CloudCreds     $SessionEX = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $CloudCreds -Authentication Basic -AllowRedirection Import-PSSession $SessionEX     Add-AzureAccount -Credential $CloudCreds Select-AzureSubscription -SubscriptionName "AzureSubscription" Login-AzureRmAccount -Credential $CloudCreds Import-Module "C:\Program Files\SharePoint Online Management Shell\Microsoft.Online.SharePoint.PowerShell\Microsoft.Online.SharePoint.PowerShell.dll" Connect-SPOService -Url https://tenant1-admin.sharepoint.com -credential $CloudCreds Clear-Host     },     "Control+Alt+1" ) ``` So now you have secured your credentials for your PS sessions. Stay tuned as I'm going to blog about how to secure the credentials even more... --- title: "Azure mobile portal - a sneak peak" url: https://bisser.io/archive/000-azure-mobile-portal---a-sneak-peak/ markdown: https://bisser.io/archive/000-azure-mobile-portal---a-sneak-peak/index.md date: 2017-05-11 lastmod: 2017-05-11 status: legacy reading_time_min: 2 --- At the [Microsoft Build 2017](https://build.microsoft.com/) there were a lot of new features announced. One, which was quite innovative was the release of the Azure Mobile Portal. With this app it's now possible to manage your Azure resources (and I mean all your Azure resources) from your smartphone or tablet via an app. So here are my first tests with the new mobile portal app. So after starting the app and selecting your subscription, you can browse all your Azure resources, which have been created in that subscription and click on each of them to get the details: ![](/static/img/azure_mobile_portal01.png) You can also filter the resources by category, for example by virtual machines: ![](/static/img/azure_mobile_portal02.png) So now let's browse into a VM's details and see what we can do: ![](/static/img/azure_mobile_portal03.png) And this is pretty cool, because at the bottom you can start and stop the VM, but you can even connect to it via RDP, so let's do this. ![](//static/img/azure_mobile_portal04.png) When it's started, we'll connect to it: ![](/static/img/azure_mobile_portal05.png) Et voila, the app store opens in order to download the Microsoft RDP app: ![](/static/img/azure_mobile_portal06.png) ![](/static/img/azure_mobile_portal07.png) And after downloading the app, you can RDP into your Azure VMs from the Azure mobile portal app straight away - pretty awesome! But wait, there is an important note about stopping the VM via the mobile portal app. If you click on stop, the VM gets only stopped, but not deallocated, which means, that you still get charged for the VM: ![](/static/img/azure_mobile_portal08.png) So be careful, when managing your VMs from the mobile portal app. For the moment you need to stop it via the Azure portal, in order to avoid unnecessary charges guys… Stay tuned for more helpful information about the Azure mobile portal, because there is more to come, look here: ![](/static/img/azure_mobile_portal09.png) --- title: "Azure Cloud Shell released - first thoughts and previews" url: https://bisser.io/archive/012-azure-cloud-shell-released---first-thoughts-and-previews/ markdown: https://bisser.io/archive/012-azure-cloud-shell-released---first-thoughts-and-previews/index.md date: 2017-05-10 lastmod: 2017-05-10 status: legacy reading_time_min: 2 --- This was one of my most awaited features, which should be unveiled at the Build2017 - the Azure Cloud Shell. And now, only a couple of hours later, it's already here and can be used do to cool stuff within the Azure portal in your favourite internet browser. To start it, simply click on the PowerShell-like icon on the upper right corner and you'll see the following: ![](/static/img/azure_cloud_shell_01.png) Now, you only need to select your subscription, where the according storage account and file share will be created. This enables you to persist your $Home directory in order to use it session-independently, which is pretty cool. This let's you work with the same environment everywhere and everytime - the only thing you need from now on is a browser and nothing else. After you have selected it, you have to wat for the initialization to be finished and… ![](/static/img/azure_cloud_shell_02-768x381.png) … now you can run your commands from the Azure CLI and Linux based commands in your Azure Cloud Shell, like listing your VMs: ![](/static/img/azure_cloud_shell_03-768x339.png) For a full list of Azure CLI commands, you can take a look at [https://docs.microsoft.com/en-us/cli/azure/](https://docs.microsoft.com/en-us/cli/azure?WT.mc_id=AZ-MVP-5003073). But what's even cooler is this - Create a resource group: ![](/static/img/azure_cloud_shell_04-768x189.png) And create your first VM within the Azure Cloud Shell: ![](/static/img/azure_cloud_shell_05-768x167.png) But that's not all, because you can even connect to that VM via SSH within the same Shell: ![](/static/img/azure_cloud_shell_06-768x140.png) So this is just awesome, as you do not need any local resources other than a browser in the future. And what's even cooler is that PowerShell support in the Azure Cloud Shell is coming too: ![](/static/img/azure_cloud_shell_08.png) So stay tuned for more to come, as I will keep you up to date on any new features in the Cloud Shell and I will also give you some ideas, which cool and helpful things can be done with this awesome tool… --- title: "A guide on simplifications for O365 admins - part I - Chrome profiles" url: https://bisser.io/archive/011-a-guide-on-simplifications-for-o365-admins---part-i---chrome-profiles/ markdown: https://bisser.io/archive/011-a-guide-on-simplifications-for-o365-admins---part-i---chrome-profiles/index.md date: 2017-05-02 lastmod: 2017-05-02 status: legacy reading_time_min: 2 --- In my career as an O365 consultant, I always had the challenge of login successfully to the different O365 tenants from our customers. As we have quite many O365 customers by now it's not that easy to remember all the passwords (but therefore we have a password management solution) but also to open a new private browsing session for the customer in order to login and don't get redirected to our ADFS, where I get authenticated via SSO. And that could be a real nightmare, when you try to open up your favourite browser and try to access the O365 portal with different credentials than your primary work account. The only escape for this problem is to open a private or incognito session and type in username and password everytime. But not anymore! Luckily a colleague of mine told me a trick on how to overcome this issue. The answer to the problem is custom Chrome profiles. This is not a big deal to establish. The only thing you need to do is to click on your Google account in the upper right corner of your Chrome window. ![](/static/img/chrome_profile01.png) After that you have the option to manage people, which allows you to add a person. ![](/static/img/chrome_profile02-300x262.png) ![](/static/img/chrome_profile03-768x604.png) Now you only need to enter a name for that person, set your starting pages (like [https://portal.office.com](https://portal.office.com) and [https://portal.azure.com](https://portal.azure.com)) and login to that pages, save your credentials and click "keep me signed in". Now, when you start your machine the next time and, you can simply click on the chrome profile, which you want to use and voila, your credentials are saved and you don't need to copy/paste username and password anymore. So that was the first part of a series on simplifactions for O365 admins, stay tuned for the next guide! --- title: "Sync Visual Studio Online items to Outlook task list" url: https://bisser.io/archive/010-sync-visual-studio-online-items-to-outlook-task-list/ markdown: https://bisser.io/archive/010-sync-visual-studio-online-items-to-outlook-task-list/index.md date: 2017-04-29 lastmod: 2017-04-29 status: legacy reading_time_min: 6 --- As a consultant, working in a team of 6 people, task management is a crucial part in my daily business. In my team, it is very important to have a good task management, in order to decide about who does the workshop for a customer next week, or who can do the conception phase during the next few weeks. For this reason we decided to use Visual Studio Online (VSO) in order to manage our work items and keep an eye on the team members' workloads, although we're no developers or engineers, who need to manage code. But the point is that there is one major part missing, when managing a team's tasks in VSO - the Outlook integration. Because personal tasks are something, which are managed in Outlook, to organise the daily work based on the calendar and the open tasks. But the missing point is that there is no native integration of VSO tasks and work items with Outlook. So many people I know overcome this issue by creating a task in Outlook everytime they get a new work item assigned in VSO. But this is too time consuming in my opinion. Therefore, I decided to search for an alternative and tried some the 3rd party addins for Outlook, which are designed to intergrate VSO in the Outlook client. But I have not found an addin, which is performant and which also makes my VSO tasks available on my mobile device. And with the releas to Microsoft's To-Do, I decided to do it on my own and create my VSO to Outlook task sync engine. And this was quite easy (and as I'm not a developer you can do it too, im sure). My goal was that I didn't want to setup any local resources or servers, but to keep it as simple as possible. So I decided on the following design: ![](/static/img/azure_function_design.png) So how to achieve this? First of all, we need to create an Azure Function app, with the following properties: ![](/static/img/azure_function01-300x494.png) After the function is created, navigate to the function app page and create a new function and get the function URL: ![](/static/img/azure_function02-830x99.png) Then we're ready to insert the following code, which basically does all our work: ```C# #r "Newtonsoft.Json" #r "Microsoft.Exchange.WebServices.dll" using System; using System.Net; using Newtonsoft.Json; using System.Net.Http; using System.Net.Http.Headers; using System.Text; using Microsoft.Exchange.WebServices.Data; public static async Task Run(HttpRequestMessage req, TraceWriter log) { log.Info($"Webhook was triggered!"); string jsonContent = await req.Content.ReadAsStringAsync(); dynamic data = JsonConvert.DeserializeObject(jsonContent); HttpContent requestContent = req.Content; string resource = jsonContent.Substring(jsonContent.IndexOf("resource")); log.Info("Item info: " + resource); var assignedTo = (object)data["resource"]["fields"]["System.AssignedTo"]["newValue"]; var title = (object)data["resource"]["revision"]["fields"]["System.Title"]; var description = (object)data["resource"]["revision"]["fields"]["System.Description"]; var iteration = (object)data["resource"]["revision"]["fields"]["System.IterationPath"]; var id = (object)data["resource"]["workItemId"]; string assigned = JsonConvert.SerializeObject(assignedTo); string bliTitle = JsonConvert.SerializeObject(title); string bliDescription = JsonConvert.SerializeObject(description); string bliIteration = JsonConvert.SerializeObject(iteration); string itemID = JsonConvert.SerializeObject(id); string itemUrl = "https://solvion.visualstudio.com/DefaultCollection/ICS/_workitems/edit/"+itemID.ToString(); bliIteration = bliIteration.Replace("\"", ""); bliTitle = bliTitle.Replace("\"", ""); bliDescription = bliDescription.Replace("\"", ""); string assignedNew = assigned.ToString(); string ite = bliIteration.ToString(); //Replace with your user ID string helper = ""; assignedNew = assignedNew.Replace("\"", ""); if (assignedNew == helper) { log.Info("Start creating task"); ExchangeService service = new ExchangeService(ExchangeVersion.Exchange2013_SP1); //Replace with your Exchange (Online) admin credentials service.Credentials = new WebCredentials("", ""); //Replace with your mail address service.AutodiscoverUrl("", RedirectionUrlValidationCallback); //Check if iteration is blank or if the item is already assigned to an iteration //Replace with your project's name or area path in VSO if (ite == "") { DateTime genericStartDate = DateTime.Now; DateTime genericEndDate = DateTime.Now.AddDays(7); string finalStartDate = genericStartDate.ToString("dd/MM/yyyy"); string finalDueDate = genericEndDate.ToString("dd/MM/yyyy"); Dictionary dictionary = new Dictionary(); bliDescription = itemUrl + " \n " + bliDescription; dictionary.Add("Subject", bliTitle); dictionary.Add("StartDate", finalStartDate); dictionary.Add("DueDate", finalDueDate); dictionary.Add("Body", bliDescription); //Replace with your mail address CreateTaskItem("", service, dictionary); return req.CreateResponse(HttpStatusCode.OK); } else { string[] dates = ite.Split(' '); string datenew = dates[2]; string startDate = datenew.Replace("\"", ""); startDate = startDate.Replace("(", ""); startDate = startDate.Replace(")", ""); string[] startEndDates = startDate.Split('-'); string startDateNew = startEndDates[0] + "2017"; string endDateNew = startEndDates[1] + "2017"; string finalStartDate = startDateNew.Replace(".", "/"); string finalDueDate = endDateNew.Replace(".", "/"); bliDescription = itemUrl + " \n " + bliDescription; Dictionary dictionary = new Dictionary(); dictionary.Add("Subject", bliTitle); dictionary.Add("StartDate", finalStartDate); dictionary.Add("DueDate", finalDueDate); dictionary.Add("Body", bliDescription); //Replace with your mail address CreateTaskItem("", service, dictionary); return req.CreateResponse(HttpStatusCode.OK); } } else { return req.CreateResponse(HttpStatusCode.OK); } if (data == null) { return req.CreateResponse(HttpStatusCode.BadRequest, new { error = "Please pass first/last properties in the input object" }); } } private static bool RedirectionUrlValidationCallback(string redirectionUrl) { // The default for the validation callback is to reject the URL. bool result = false; Uri redirectionUri = new Uri(redirectionUrl); // Validate the contents of the redirection URL. In this simple validation // callback, the redirection URL is considered valid if it is using HTTPS // to encrypt the authentication credentials. if (redirectionUri.Scheme == "https") { result = true; } return result; } public static void CreateTaskItem(string targetMailId, ExchangeService EXservice, Dictionary dictionary) { Microsoft.Exchange.WebServices.Data.Task task = new Microsoft.Exchange.WebServices.Data.Task(EXservice); DateTime convertedStartDate = DateTime.ParseExact(dictionary["StartDate"], "dd/MM/yyyy", System.Globalization.CultureInfo.CurrentUICulture.DateTimeFormat); DateTime convertedDueDate = DateTime.ParseExact(dictionary["DueDate"], "dd/MM/yyyy", System.Globalization.CultureInfo.CurrentUICulture.DateTimeFormat); task.Subject = dictionary["Subject"]; task.DueDate = convertedDueDate; task.StartDate = convertedStartDate; task.Body = dictionary["Body"]; task.Save(new FolderId(WellKnownFolderName.Tasks,targetMailId)); } ``` In the code I'm referencing the Exchange WebServices API. In order to use that API in your code, you have to download the API from [https://www.microsoft.com/en-us/download/details.aspx?id=42022](https://www.microsoft.com/en-us/download/details.aspx?id=42022) and install it on your machine. After installing it, you can find the file in your %ProgramFiles%\Microsoft\Exchange\ directory. Now you need to head over to the function's debug console: ![](/static/img/azure_function08-768x596.png) ![](/static/img/azure_function09-480x324.png) Now navigate to the path D:\home\site\wwwroot\ and create a bin folder: ![](/static/img/azure_function10-480x713.png) Now navigate to that bin folder and drag and drop your Microsoft.Exchange.WebServices.dll file in there. After the function is in place, we need to head over to VSO and create a service hook as follows: ![](/static/img/azure_function03-480x445.png) On the first page select "Work item updated" and your area path. On the next page insert your function's URL and hit "Test" or "Finish", as this is it on the VSO side. ![](/static/img/azure_function04-480x520.png) Now you can create a new work item (either in a sprint or in the backlog) and assign it to your user. ![](/static/img/azure_function05-768x395.png) If you have configured everything correct, you should see that the Azure function is triggered in the logs: ![](/static/img/azure_function06-768x172.png) And after a couple of seconds you should see the task in your Outlook task list (unfortunately my Office is in German - sorry guys): ![](/static/img/azure_function07-768x288.png) And whats even cooler now is this: ![](/static/img/20170429_215302000_iOS.png) The task is also showing up on my mobile device in the To-Do app :) and I can keep track of my tasks, when I'm traveling too. But this is basically it. As I already said, I'm not an engineer, so the code could be improved im pretty sure, but it does what I want it to do, so I'm fine for the moment… But if you have some improvements please let me know! Maybe I'll follow up with a second part on assigning the task to a category in Outlook as well, we'll see... --- title: "How to use Microsoft To-Do, even when there is no option of activation in the O365 tenant" url: https://bisser.io/archive/009-how-to-use-microsoft-to-do-even-when-there-is-no-option-of-activation-in-the-o365-tenant/ markdown: https://bisser.io/archive/009-how-to-use-microsoft-to-do-even-when-there-is-no-option-of-activation-in-the-o365-tenant/index.md date: 2017-04-27 lastmod: 2017-04-27 status: legacy reading_time_min: 2 --- Microsoft recently announced that its newest task management app "**To-Do**", which shall be the successor of **Wunderlist**, will soon be available. I'm not going into detail about the functionality, because this is described here: [https://blogs.office.com/2017/04/19/introducing-microsoft-to-do-now-available-in-preview/](https://blogs.office.com/2017/04/19/introducing-microsoft-to-do-now-available-in-preview/) and [https://products.office.com/en-us/microsoft-to-do-list-app](https://products.office.com/en-us/microsoft-to-do-list-app) But as Microsoft announced the preview of To-Do, many people downloaded the app and tried to synchronize their Outlook tasks into the To-Do list - without success. The problem in most cases is that you can't even login with a work or school account in the app. Most probably you'll receive one of the two error messages, which are shown below: ![](/static/img/todo_01.png) ![](/static/img/todo_02.png) However, if you take a look on the comments in [https://blogs.office.com/2017/04/19/introducing-microsoft-to-do-now-available-in-preview/](https://blogs.office.com/2017/04/19/introducing-microsoft-to-do-now-available-in-preview/) you'll notice, that almost everyone has the exact same issue. Digging a bit deeper, this link may help: [https://todosupport.helpshift.com/a/microsoft-to-do/?s=account&f=administration-of-to-do-in-office-365&l=en&p=web](https://todosupport.helpshift.com/a/microsoft-to-do/?s=account&f=administration-of-to-do-in-office-365&l=en&p=web) The point is that the support guys advise you to look at the O365 admin portal under Settings - Services & add-ins where you should see the option of activating To-Do. In my case, there is no To-Do option available: ![](/static/img/todo_03.png) But there is a hint, how you still can use To-Do with a work or school account. It's a matter of the right license. Because there are two important cases, where To-Do is not available yet: * On-Premise installations of Exchange Server and Microsoft accounts that are not on Exchange Online. * Office 365 Pro Plus, Business, Kiosk K1, Enterprise E5 without PSTN, Office 365 DE plans and Education accounts. And in my case I had an E5 license assigned. So after assigning an E3 license to my user, To-Do login with my O365 user succeeded and To-Do started working like charm. It instantly imported all my Outlook tasks and the synchronization between To-Do and my tasks in Outlook works great, as you can see below: ![](/static/img/todo_04.png) Unfortunately, the screenshots of my To-Do app are in German, as I can't change the language of the app. I hope this helps you activating your To-Do app with your O365 account. Have fun managing your tasks! --- title: "Microsoft Bot Framework - Useful resources \u0026 links" url: https://bisser.io/microsoft-bot-framework-useful-resources-links/ markdown: https://bisser.io/microsoft-bot-framework-useful-resources-links/index.md date: 2017-04-14 lastmod: 2017-04-14 tags: AI, Bot Framework categories: AI, Bot Framework description: "Microsoft Bot Framework - Useful resources \u0026 links" status: legacy reading_time_min: 1 --- As you may know, **Microsoft** has released a Framework for building **Bots** on an **Azure**. You may encounter the problem that you are thrilled to start building a bot, but you don't know how to begin and where to find the resources. So I thought of providing you with a list of useful resources and links on the Bot Framework which you may want to conduct: # Links * [Official Microsoft Bot Framework Blog](https://blog.botframework.com/author/botframeworkteam/) * [Bot Framework - Making Bots More Intelligent](https://msdn.microsoft.com/en-us/magazine/mt795186.aspx) * [Forwarding activities / messages to other dialogs in Microsoft Bot Framework](http://www.garypretty.co.uk/2017/03/26/forwarding-activities-messages-to-other-dialogs-in-microsoft-bot-framework/) * [New Bot Framework features and updates](https://blog.botframework.com/2017/02/02/new-bot-framework-features-and-updates/) * [Bots are banging: Getting Started with the Microsoft Bot Framework & Node.js](https://medium.com/the-node-js-collection/bots-are-banging-getting-started-with-the-microsoft-bot-framework-node-js-b16ccb96b32f) # Videos * [Getting Started with Building Bots with Microsoft's Bot Framework](https://www.pluralsight.com/courses/microsoft-bot-framework-getting-started) * [Microsoft Bot Framework and Cognitive Services: Make your bot smarter!](https://channel9.msdn.com/Events/TechDaysOnline/MVP-Led-TechDays-Online-February-2017/Microsoft-Bot-Framework-and-Cognitive-Services-Make-your-bot-smarter) * [DevOps for the Bot Framework](https://channel9.msdn.com/Series/DevOps-for-the-Bot-Framework) * [Telemetry for the Bot Framework](https://channel9.msdn.com/Series/DevOps-for-the-Bot-Framework/Telemetry-for-the-Bot-Framework) * [Testing the Bot Framework](https://channel9.msdn.com/Series/DevOps-for-the-Bot-Framework/Testing-the-Bot-Framework) # Code Samples * [Embeddable web chat control for the Microsoft Bot Framework](https://github.com/Microsoft/BotFramework-WebChat) * [Forwarding activities / messages to other dialogs in Microsoft Bot Framework](http://www.garypretty.co.uk/2017/03/26/forwarding-activities-messages-to-other-dialogs-in-microsoft-bot-framework/) --- title: "Get the storage location of mailboxes in Exchange Online" url: https://bisser.io/archive/007-get-the-storage-location-of-mailboxes-in-exchange-online/ markdown: https://bisser.io/archive/007-get-the-storage-location-of-mailboxes-in-exchange-online/index.md date: 2017-04-10 lastmod: 2017-04-10 status: legacy reading_time_min: 2 --- Microsoft always claims, that all data from a customer is only stored in that region, where the Office 365 contract is bound to. Now that means, that all SharePoint Online data and Exchange Online data will only be stored in the EMEA datacenters, when the contract is bound to Microsoft Austria for example. Now if you take a look at [http://o365datacentermap.azurewebsites.net/](http://o365datacentermap.azurewebsites.net/) there is already quite a big amount of datacenters existing (especially in the US and in EMEA). The point now is that, with the fast growth of Microsoft datacenters, customers do not know where their data is really stored, as it could make a difference for a company wether their mailboxes are stored in Austria or in Amsterdam or in Finland (due to legal restrictions). So many IT admins ask the question: "How can I check where my mailbox data is actually stored?" And I finally have come up with an answer to that question. In order to check where the O365 mailbox data is stored you can download and execute my PowerShell script from [here](https://gallery.technet.microsoft.com/ExO-PowerShell-Get-storage-612c28fb) and execute as follows to check all mailboxes in your O365 tenant: ```powershell .\exo_check_mailbox_location.ps1 -username admin@bisser.at -password "Pa$W0rd" ``` But sometimes you have the requirement that only some of your mailboxes need to be stored in a specific region due to legal restrictions. Therefore, you can use the same script with the following paramters: ```powershell .\exo_check_mailbox_location.ps1 -username admin@bisser.at -password "Pa$W0rd" -primarySMTPAddress stephan@bisser.at ``` After running this script you will get an output which is similar to the following: ![Mailbox](/static/img/mailbox.png) So this script can be run on a scheduled base in order to check where your data is really stored, if you have some legal restrictions, which need to be taken into consideration when using Office 365... --- title: "Cognitive Services - Vision Category" url: https://bisser.io/cognitive-services-vision-category/ markdown: https://bisser.io/cognitive-services-vision-category/index.md date: 2017-03-21 lastmod: 2017-03-21 categories: AI, Cognitive Services description: "Cognitive Services - Vision Category" status: legacy reading_time_min: 3 --- ## Computer Vision Most of us will quickly bring image analysis in conjunction with high mathematical formulas and pattern recognition algorithms. The **Computer Vision API** but is based on a completely different level of abstraction. If one sends a picture of a squirrel on the service, "a squirrel is sitting on a rock" and a percentage chance comes in response, how much you can trust the statement. In addition, their position, as well as dominant color values can be queried tags and categories (animal, outdoor etc.), including faces. For portals, where images are uploaded by users, marking of potentially racist or adult content is helpful. In addition to the - almost unspectacular in comparison - **OCR recognition** (optical character recognition), which is also offered as a service, a very useful API stands out: the **thumbnail service**. From an arbitrary image, a thumbnail of the desired size is generated and it tries to choose the best cut-out. Detected a logo, is the image fully preserved and will only shrink. Photos faces and dominant objects as well as possible in the image remain, while the rest will be truncated. ## Emotion API Who was already excited by the recognition of the age and sex, should have seen the E**motion API**. Faces can be analyzed in terms of feeling expression: _Does the person look angry, shocked, scared, happy, neutral, sad or_surprised? The service works for pictures the same as for videos and is also available in a limited free version available (thirty thousand image requests, three hundred uploads per month). The premium packages are between 0.10 and 0.25 dollar per thousand transactions. ## Face API The F**ace API** cares less about the expression as to the identification of faces. While earlier, the position of the face is detected, this API also provides information about the eyes and eyebrows, nose and corners of the mouth. It is recognized that the person wears a beard or is a glasses on has. This data is in turn relevant to **identify faces**. The face API can determine whether two photos show the same person or not - and how inferior everywhere with a probability. The detection works doing surprisingly well, even if the faces are partially covered or look into the camera. ## Video API The vision categoriy is finally completed by some video services. From image stabilizers and detecting faces, already described a thumbnail service to determine of possible meaningful thumbnails available is also here. In addition to using "**Motion**detection" sequences are detected, in which a movement takes place - whether it reliable is enough for the homemade alarm system, you need to decide. --- title: "Check if domain is already registered/verified in an Office 365 tenant" url: https://bisser.io/archive/005-check-if-domain-is-already-registered---verified-in-an-office-365-tenant/ markdown: https://bisser.io/archive/005-check-if-domain-is-already-registered---verified-in-an-office-365-tenant/index.md date: 2017-03-08 lastmod: 2017-03-08 status: legacy reading_time_min: 4 --- As a consultant dealing with Office 365 onboarding projects, I got one question quite often from my customers and partners: **_“Is it possible to check, if my domain is already registered in an Office 365 tenant?”_** The reason, why I often get asked this question is that, many IT departments in most companies already did some research on Office 365 and the IT admins also often create a test tenant in order to check, what features are available and what can be done with O365\. But that’s not a problem at all. The problem raises if the IT guys add their productive domain (e.g. bisser.at) to that test tenant and verify it as well. When they do that, they often play around a little and then forget about that until the management says that it’s time to go for O365, where I step into the game. Usually we then have some workshops and so one and create then a brand new O365 tenant in order to start using the cloud. There comes the first question: _“How shall we call our tenant?”_ – which can take some time to be answered, especially, when you want to create a tenant with your company’s name, which is already existing. Luckily,Aaron Dinnage (Solution Specialist at Microsoft) came up with a solution in order to check if the preferred tenant name already exists. You can read more [here](https://t.co/RlSYyl6eV8) After the tenant is created with the right name the next problem raises: _“Why can’t I add my domain to that tenant?”_ and the answer is clear: **_“Because it is already registered to a tenant.”_** But this is not the only reason, why I tried to come up with a solution to this question. The other reason is the troubleshooting aspect. When you invite an external person to your O365 tenant with his or her proper mail address, Azure AD could cause sign in problems. The reason for that is that if you invite a person who doesn’t know about O365, the person just clicks on the invitation link in the mail and follows the instructions (setup account, create password, etc.) because the person is not aware, that he maybe already has an O365 user which has the same logon name as the mail address you invited. Therefore, the login process will fail and as a result the person calls you and says “I can’t login, please help me!”. And you as an admin then try to troubleshoot the problem leading to the fact that you will reset the password, send another invite and so on – always with the same result. In order to avoid these pesky troubleshooting hours, my script shall come up with a solution, which you may think now is quite obvious: The user is already registered in an AAD tenant with the exact same mail address and must therefore login with these credentials. In order to check if there is an AAD tenant already registered with that (SMTP) domain, you can use my script, which can be found [here](https://gallery.technet.microsoft.com/Check-domain-registration-2fb6d097) When you execute the script, you will have to pass 1 (or 2 params): * Domain: This represents the domain you want to check against a possible O365/AAD registration * UseAnotherAccount: This is an optional param, where you can use the option “Use another account” within the O365 login process, if you have cached some O365 credentials in the Internet Explorer So to run the script and check your domain, execute the following statement in a PowerShell window where you have stored that script: ```powershell .\o365_check_registered_domain.ps1 -domain "bisser.at" ``` Or by using “Use another account”, you need to run the following: ```powershell .\o365_check_registered_domain.ps1 -domain "bisser.at" -useAnotherAccount $true ``` After you execute this one, the PS will open up an IE window, which is not visible to you, navigating to [https://portal.office.com](https://portal.office.com/) and will try to insert a dummy user with that domain ending, in order to check if the domain is a known domain to Microsoft or not: ![](/static/img/portal_O365.png) If the domain is already registered in O365/AAD, you will see the following output in your PS window: ![](/static/img/PS_domain01-768x97.png) But if the domain is not registered in an O365/AAD environment, you’ll see the following output, which means, that you’re good to go with that domain: ![](/static/img/PS_domain02-768x108.png) As this is the first version of my script, I will keep updating this one and improve it to make troubleshooting and checking of domain registration more easy, so keep reading to get updates on this one… --- title: "Office 365: Merge AD user with AAD contact via Azure AD Connect" url: https://bisser.io/archive/004-office-365-merge-ad-user-with-aad-contact-via-azure-ad-connect/ markdown: https://bisser.io/archive/004-office-365-merge-ad-user-with-aad-contact-via-azure-ad-connect/index.md date: 2017-02-22 lastmod: 2017-02-22 status: legacy reading_time_min: 3 --- Many companies, which use Office 365, especially Exchange Online, often do one thing during the onboarding phase: The companies import contacts into their Office 365 tenant, in order to feed the Global Address  List (GAL) with users and contacts, which the already onboarded users can then address via the GAL. This setting is also helpful, when a multinational company decides to go for Office 365 (Exchange Online), where all users are not in one single Active Directory, but there is a multi-forest AD construct established. The migration or onboarding of all users is usually not a task within a couple of weeks, but months or years. Therefore, the use of contacts, which are imported into the Office 365 tenant, is useful for the already migrated users, so that the GAL is already populated from the beginning on during the migration process. This is always a requirement for the management board, because these kind of users are not in favor of remembering the contact information of all their colleagues. The administrators usually have to fulfill the following steps for creating contacts from user objects originated in a different AD forest, which cannot be synchronized at first: * Export user objects and relevant attributes from AD forests * Import these objects as MailContacts (via O365 admin portal or PowerShell – New-MailContact) When the contacts are created in the O365 tenant and the users then have to be synchronized from the local AD, this will cause an error in Azure AD Connect. But why? SMTP match is a supported method of joining a local user object with a cloud object, if the SMTP address matches (which is usually the case when creating contacts in O365). But there is a significant problem: The type of the contact, which was created in O365 – the MailContact This is the default object type, when a contact is created via the O365 admin portal. The following graphic should illustrate the problem: [![](/static/img/aadc_user_contact_merge_error.png)](/static/img/aadc_user_contact_merge_error.png) In the AADConnect synchronization service, you will get the following error message: “InvalidSoftMatch”, which is described here: [https://docs.microsoft.com/en-us/azure/active-directory/connect/active-directory-aadconnect-troubleshoot-sync-errors](https://docs.microsoft.com/en-us/azure/active-directory/connect/active-directory-aadconnect-troubleshoot-sync-errors?WT.mc_id=AZ-MVP-5003073) So what is the solution to this? The answer is easy: Replace the MailContact objects with MailUser objects. This can either be done by GUI or automated via PowerShell (Remove-MSOLContact followed by New-MailUser). This leads to the fact that if the user then gets created in the local AD and therefore gets synchronized via the Azure AD Connect service, the AD user gets merged via the SMTP soft-match mechanism and the MailUser in the AAD gets converted to a synchronized AD user: [![](/static/img/aadc_user_contact_merge_works-350x581.png)](/static/img/aadc_user_contact_merge_works-350x581.png) So therefore: heads up when creating contact objects, if the contacts represent users, which eventually get synchronized via AADConnect sooner or later. --- title: "Restrict Office 365 group creation – the “hidden” feature" url: https://bisser.io/archive/003-restrict-office-365-group-creation--the-hidden-feature/ markdown: https://bisser.io/archive/003-restrict-office-365-group-creation--the-hidden-feature/index.md date: 2017-02-17 lastmod: 2017-02-17 status: legacy reading_time_min: 2 --- As Office 365 groups are now more and more used in various Office 365 tenants, many administrators complain about a restriction setting. The problem is, that you can only allow or disable the creation of Office 365 groups for all users of your tenant. From an administrative point of view, this setting would be nice to have, because at the moment Microsoft is pushing groups but if you as an admin can’t control who is able to create groups and which groups can be created, this is nothing but a nightmare. Additionally, if a user creates an O365 group with the e-mail address sales@company.com and you have to create an AD security group with the e-mail address sales@company.com, this would lead to a synchronization error in Azure AD Connect, saying, that there exists already an object with the e-mail address sales@company.com. You as an admin would then go to your AD and search for that object, resulting in nerve-stretching minutes, because you can’t find that object – of course not – because it only exists within your O365 tenant. So in order to to restrict, which users should be allowed to create groups, the following PowerShell CmdLets will help you to achieve this goal. First of all, you have to install the Azure AD Powershell V1.1.130.0 public preview from [here](http://connect.microsoft.com/site1164/Downloads/DownloadDetails.aspx?DownloadID=59185) in order to have the right commands available. Then connect to your O365 tenant and connect to your Exchange Online environment within your PS and execute the following lines of code: ```powershell Get-MsolCompanyInformation Set-MsolCompanySettings -UsersPermissionToCreateGroupsEnabled $True ``` Then you would have to create a normal security group (in your AD or in cloud only) and add the members to that group, which should be able to create O365 groups. In my case this group is called “O365_Group_Creation”. After that exceute the following commands in your PS session: ```powershell $grp = Get-MsolGroup -SearchString “O365_Group_Creation” $setting = Get-MsolAllSettings | Where-Object { $_.DisplayName -eq “Group.Unified” } $settingId = $setting.ObjectId $value = $setting.GetSettingsValue() $value[“GroupCreationAllowedGroupId”] = $grp.ObjectId $value[“EnableGroupCreation”] = $false Set-MsolSettings -SettingId $settingId -SettingsValue $value ``` After you have executed the commands successfully, I would advise to test the creation of O365 groups with a user who is member of that security group (which should work) and with a user who is not member of that group (which should fail). --- title: "Cognitive Services - What is that?" url: https://bisser.io/cognitive-services-what-is-that/ markdown: https://bisser.io/cognitive-services-what-is-that/index.md date: 2017-01-19 lastmod: 2017-01-19 categories: AI, Cognitive Services description: "Cognitive Services - What is that?" status: legacy reading_time_min: 1 --- **Microsoft** just published a new [article](https://docs.microsoft.com/en-us/azure/cognitive-services/welcome?WT.mc_id=AI-MVP-5003073) on the topic of **Cognitive Services** (formerly**Project Oxford**). The Cognitive Services are a collection of SDKs and APIs along with services which should make your life easier if you want to build an intelligent application. These services basically leverage Microsoft's existing platforms and services like Machine Learning and so on to enable developers to create applications with intelligent features like the following: * **Emotion Detection** * **Video Detection** * **Speech Detection** * **Face Recognition** * **Speech Recognition** * **Vision Recognition** * **Language Understanding** * **Speech Understanding** >Our vision is for more personal computing experiences and enhanced productivity aided by systems that increasingly can see, hear, speak, understand and even begin to reason. The services are all hosted in Azure datacenters to ensure high performance and ease of use. You can start integrating these nice APIs and SDKs as Microsoft offers you to start with a [free trial](https://azure.microsoft.com/en-us/try/cognitive-services/) with the ability to upgrade it for production use later on. As all the APIs have a free trial plan to start with you can basically explore each service to find out if that fits your application and if it has the features you need to deploy an intelligent application… --- title: "Azure AD Connect to sync users and groups from AAD to local AD" url: https://bisser.io/archive/001-azure-ad-connect-to-sync-users-and-groups-from-aad-to-local-ad/ markdown: https://bisser.io/archive/001-azure-ad-connect-to-sync-users-and-groups-from-aad-to-local-ad/index.md date: 2017-01-07 lastmod: 2017-01-07 status: legacy reading_time_min: 2 --- As **DirSync** and **Azure AD Sync** will soon be not supported anymore, you should migrate your old DirSync Server to the new Azure AD Connect service. And the**Azure AD Connect** tool, which is the successor of the Azure AD Sync Service has a couple of new and cool features. One of these features can develop to a real killer feature in some enterprises – Sync cloud users and groups to the on-premise Active Directory. As with the new Office 365 Groups a new group type joins Distribution and Security Groups in the cloud, the ability to sync these cloud groups back to the local AD, the enterprises do not have to go without the Office 365 groups in their local AD. Unfortunately the Office 365 group type is not part of the Microsoft Active Directory yet, but the Office 365 Groups, which are synchronized to the local AD are transformed to normal Distribution Groups, which then in turn, can be used in an on-premise Exchange deployment. In the future, these groups will may be integrated in the on-premise Active Directory and will replace the Security and Distribution Groups. The same feature was published for users. Office 365 customers have now the ability to synchronize users, who were created in the cloud, to the local AD and use them for their on-premise applications. This leads to the fact, that you can create and manage users everywhere and do not need to be connected to your on-premise domain controller. Unfortunately you can, at the moment, only select one organizational unit to which the users get synchronized. But as these features get updated in the future, I will provide you with detailed information as soon as the updates get announced, so stay on reading…